Prof. Jean Cleymans, head of the SA-CERN consortium, visits Wits

Professor Jean Cleymans, head of the SA-CERN consortium, visits the School of Physics of the University of the Witwatersrand. Wits is actively involved in CERN activities contributing to the four   directions in the consortium: ALICE, ATLAS, Isolde and Theory.

Prof. Cleymans visited the various facilities used at Wits for CERN-related activities. This includes synergistic activities with Nuclear and Radiation physics, Material Sciences, Electrical and Information engineering. Overall, over 50 people are involved in CERN activities, ranging from undergraduate, graduate students, post-doctoral fellows, to technical and academic staff. Most of people involved in the program are students.

After visiting the facilities we had a meeting with people involved with the CERN programme at Wits. Below is a picture that includes Professor Joao Rodrigues, the Deputy Dean of the Faculty of Sciences and former Head of the School. Dr Xifeng Ruan, Dr Luis March and Kehinde Tomiwa were connected remotely. Missing in the picture are Prof. Zeblon Vilakazi, two post-doctoral fellows at ALICE, a PhD student from Isolde, Vincent Govender, Jonathan Padavatan,  and ATLAS students Shell-may Liao, Guillermo Hamity, Titus Masike, Marc Sacks and Kamela Sekonya, among others:

Prof. Jean Cleymans and Prof. Bruce Mellado addressing the audience. In the background, Dr. Xifeng Ruan, Dr. Luis March, and Kehinde Tomima, based abroad.

Some of the Wits academic staff involved at SA-CERN with Prof. Jean Cleymans. From left to right, Prof. Elias Sideras-Haddad, Dr. Deepak Kar, Prof. Deena Naidoo, Prof. Bruce Mellado, Prof. Alan Cornell.

First Run 2 ATLAS results with strong Wits contribution

The first results from LHC Run 2 was shown at the European Physical Society Meeting in Vienna last week. While we are excited about the next discovery with the new data, the first task with a new collision energy data is always to make sure we understand our detector performance and verify that the Standard Model predictions are correct.

The former was very important because the ATLAS detector went through a fair bit of improvement during the long shutdown. A new tracking detector (the insertable B-layer, or the IBL) was added next to the beam pipe, which dramatically improved the accuracy of the track reconstruction and the identification of jets originating from bottom quarks, which is important for many searches. Shown below is the event display of a proton-proton collision event recorded by ATLAS on 3 June 2015, with the first LHC stable beams at a collision energy of 13 TeV.

The Monte Carlo event generators are extensively used in measurements and searches. While the perturbative part is modelled based on very accurate theoretical prediction, the non-perturbative part is constrained by measurements. So it was crucial to test the phenomenological energy extrapolation models used in these generators by looking at the 13TeV data. The usual "minimum-bias" and "underlying event" measurements were performed, where different distributions with charged particles were measured, and the models seemed to be in reasonable agreement with data. This gave us confidence that we are ready to use these models for our exciting new physics searches. Dr. Deepak Kar, who joined the Wits group recently, played a major role in getting this results public in a such a short time as the ATLAS soft-QCD subgroup convener and an analyser.

The figure below shows the the activity in an event which is mostly not coming from the hard collision, i.e our process of interest. This is termed underlying event, and represents the soft background which needs to be modelled well by the generators. The 13 TeV data is compared to several such models, and the ones to be used by ATLAS describe the data reasonably well.

More details can be found at the following ATLAS briefing.

Memorial Service for Miss Kamogelo Vilazaki

Our community has been shocked by the sudden death of the eldest daughter of Professor Zeblon Vilakazi, Deputy Vice Chancellor for Research and Postgraduate Affairs of the University of the Witwatersrand. Zeblon is a colleague, a leader and a friend. We stand by him in these excruciating moments.

The memorial was a powerful demonstration of love, fortitude, humility and empathy in the face of a terrible loss. 

Setting up a virtual machine for data analysis at the LHC

Analysis? I thought you only posted about electronics?

The LHC is now producing data as 50ns bunch crossing and is getting ready to bring that down to 25ns! This means physics analyses are picking up and more people are getting into it. I have been involved in the double Higgs production decaying to di-photon and di-bquark objects. This has involved the development of an analysis framework and more recently the statistical tools associated with it.

Why this post and why now?

I have been working on the analysis code for over 7 months now. Why am I suddenly posting a how-to? Well, my Virtual Machine packed up for some reason and I have to re-install everything so a nice recipe will help me remember things if it happens again as well as for future students and possibly even you.

Lets get started!

I am using VMPlayer as my virtual machine. It doesn't matter what you use but I prefer this over virtual box because of the nice tools that come with VMP and I am comfortable with it and its free. You can download it here: https://www.vmware.com/products/player

First up you need to download the ISO of choice to install on your VM. I chose SLC6.6 since it was the latest at the time of writing and its CERN software so I know the analysis code will work on it. Up to you what you wish to use. Here is the link to install SLC6. Try use the 64bit version: http://linux.web.cern.ch/linux/scientific6/docs/install.shtml

I have not tried SLC7. I am too scared to move to CentOS :P

So, once you have downloaded the SLC6.6 64bit ISO we can get started.

Open VMPlayer and create a new machine. Use the ISO as the input disk. I wont go into much detail about the install as its fairly straight forward and you can follow the link.

First time booting up

After putting in a root password we should move onto creating a user. This is where we will create the main user of the VM. Its a good idea to make the user have the same name as your CERN account. This will make life a lot easier when sshing to lxplus or using the GRID etc. On the Create User page fill in your CERN NICE account name. Choose any password. Next.

You should have booted up into SLC6. Now we need to add your user into the wheel group so you can use sudo.

su -
nano /etc/sudoers

uncomment the # %wheel line by removing the "#"

## Allows people in group wheel to run all commands
%wheel ALL=(ALL)       ALL

Add yourself to the wheel group:

sudo usermod -aG wheel johnsmith
exit

Test sudo works:

sudo ls

Update!

Now that you have sudo rights. Lets update the system. This can take long if you are using a remote package repository. If you are in South Africa you can find the mirror repo for SLC at www.mirror.ac.za by scrolling to the bottom. Adding a repo is something you can google :)
The update may take a while so go grab some coffee while this updates.

sudo yum update

Install VMware-tools

Player->manage->Install VMware Tools

An ISO should appear on the Desktop

It should either open automatically. If not just google "mount iso in linux"

Extract the .tar.gz file to the desktop.

Open a console and cd into the folder you just extracted
Run the vmware-install.pl executable as sudo
sudo ./vmware-install.pl

I leave everything on default. Now you can drag/drop copy/paste from the host to the VM.

Installing software collections from CERN

We have a basic working desktop. Now we want to make sure we have an up to date compiler. We make use of the CERN software collections: http://linux.web.cern.ch/linux/scl/#scl12

Install the packages:

sudo yum install python27 devtoolset-3

For some reason python33 doesnt work with the rcSetupTool we install later so stick to 27 for now. We want to enable these packages when ever we start a new bash. So in ~/.bashrc add the following:

source /opt/rh/devtoolset-3/enable
source /opt/rh/python27/enable

Enable the .bashrc by sourcing it then check the version:

source ~/.bashrc
g++ -v

and you should see 4.9. For the new analysis code you NEED 4.8 or above. This is important.

DO NOT COMPILE ANYTHING UNTIL THIS IS DONE!!!

SERIOUSLY.... I think the default is g++ v4.3 which is ancient!

Installing ROOT

If you have done any data analysis you probably have heard of the ROOT package. The latest version can be found here: https://root.cern.ch/drupal/content/downloading-root

At the time of writing Root-v6.04.00- was not compatible with Analysis Base so stick to the older Root-v6.02.12

After you have downloaded the source code we need to install all the pre-requisites: https://root.cern.ch/drupal/content/build-prerequisites

sudo yum install git make gcc-c++ gcc binutils libX11-devel libXpm-devel libXft-devel libXext-devel

Optional packages: 

sudo yum install gcc-gfortran openssl-devel pcre-devel mesa-libGL-devel glew-devel ftgl-devel mysql-devel fftw-devel cfitsio-devel graphviz-devel avahi-compat-libdns_sd-devel libldap-dev python-devel libxml2-devel gsl-static

I normally create a root-src and a root-build directory. This lets me manage which version I am using.

Extract the root source into the root-src directory.

From inside the directory:

sudo ./configure --all
sudo make -j 4

Then define the install location which should be the root-build directory

export ROOTSYS=/path/to/install/root-builds/
sudo make install

Installing Boost

Boost is a set of peer reviewed libraries with general functionality. You can download it here: http://sourceforge.net/projects/boost/files/boost/1.58.0/

To install it:

tar -xvjf boost_1_58_0.tar.bz2
cd boost_1_58_0/
sudo ./b2
sudo ./b2 install
sudo ./bootstrap.sh --prefix=/usr/local/boost-builds/v1-58-0/ --with-libraries=system,regex,filesystem,program_options,test

Now we tell the environment where to find Boost:

export BOOSTLIBDIR=/usr/local/boost-builds/v1-58-0/lib
export BOOSTINCDIR=/usr/local/boost-builds/v1-58-0/include

Getting things ready for the analysis

So now our machine is almost ready for the data analysis. We have ROOT and Boost libraries installed. Now we just need to do some fine tuning to make our lives easier for updating the analysis code.

A large amount of this is taken from https://twiki.cern.ch/twiki/bin/view/AtlasComputing/SoftwareTutorialxAODEDM#On_your_local_machine_optional

rcSetupLocal

rcSetup is the tool which manages the analysis base. The base is a collection of tools which are used in all areas of an analysis. Calibrations, selections, etc etc. 

First we need Subversion:

sudo yum install svn

Lets set it up as follows:

mkdir -p ~/ATLAS/sw/rcSetup
cd ~/ATLAS/sw/rcSetup/
svn co svn+ssh://svn.cern.ch/reps/atlasoff/rcSetup/tags/rcSetup-00-04-09
ln -s rcSetup-00-04-09 latest

In our .bashrc file we will create a command which is not the default rcSetup but rather rcSetupLocal (To distinguish between the two):

rcSetupLocal() {
    export rcSetupSite=~/ATLAS/sw/releases
    export PATHRESOLVER_ALLOWHTTPDOWNLOAD=1
    source ~/ATLAS/sw/rcSetup/latest/rcSetup.sh $*
}

Setting up Kerberos

We will be checking out packages using SVN. We will be checking out a LOT of packages and since rcSetup is a wrapper around SVN using SSH we end up having to type passwords multiple times for a single download. Fine if there are two or three but we will be downloading hundreds. So we need to setup the kerberos to fetch a ticket from CERN which will give us an extended period of passwordless use.

You should have kerberos installed already. Check if you have

/etc/krb5.conf

If yes then replace the contents with

; AD  : This Kerberos configuration is for CERN's Active Directory realm.
;
; /etc/krb5.conf
; On SLC nodes this file is maintained via ncm-krb5clt(1), local changes may be lost.
; If you need to add your realm, look at the "template" file 
; in /usr/lib/ncm/config/krb5clt/etc_krb5.conf.tpl
; or get in touch with project-elfms@cern.ch
;
; Created   1-Apr-2011
; Modified  3-Mar-2014
;

[libdefaults]
 default_realm = CERN.CH
 ticket_lifetime = 25h
 renew_lifetime = 120h
 forwardable = true
 proxiable = true
 default_tkt_enctypes = arcfour-hmac-md5 aes256-cts aes128-cts des3-cbc-sha1 des-cbc-md5 des-cbc-crc
 allow_weak_crypto = true

[realms]
 CERN.CH = {
  default_domain = cern.ch
  kpasswd_server = cerndc.cern.ch
  admin_server = cerndc.cern.ch
  kdc = cerndc.cern.ch

  v4_name_convert = {
     host = {
         rcmd = host
     }
  }
 }
; the external institutes info is completely static for now and comes
; straight from the NCM template
 FNAL.GOV = {
  default_domain = fnal.gov
  admin_server = krb-fnal-admin.fnal.gov
  kdc = krb-fnal-1.fnal.gov:88 
  kdc = krb-fnal-2.fnal.gov:88 
  kdc = krb-fnal-3.fnal.gov:88 
 }
 KFKI.HU = {
  kdc = kerberos.kfki.hu
  admin_server = kerberos.kfki.hu
 } 
 HEP.MAN.AC.UK = {
  kdc = afs4.hep.man.ac.uk
  kdc = afs1.hep.man.ac.uk
  kdc = afs2.hep.man.ac.uk
  kdc = afs3.hep.man.ac.uk
  admin_server = afs4.hep.man.ac.uk
  kpasswd_server = afs4.hep.man.ac.uk
  default_domain = hep.man.ac.uk
 }
[domain_realm]
 .cern.ch = CERN.CH
 .fnal.gov = FNAL.GOV
 .kfki.hu = KFKI.HU
 .hep.man.ac.uk = HEP.MAN.AC.UK
[appdefaults]
   pkinit_pool =  DIR:/etc/pki/tls/certs/
   pkinit_anchors = DIR:/etc/pki/tls/certs/
; options for Red Hat pam_krb5-2
 pam = {
   external = true
   krb4_convert =  false 
   krb4_convert_524 =  false 
   krb4_use_as_req =  false 
   ticket_lifetime = 25h
 }

If no then install Kerberos and then replace the file.

Test the code by typing:

kinit

You should be prompted for your CERN username and password. If you do not have the same username then it should query you or else visit here: http://linux.web.cern.ch/linux/docs/kerberos-access.shtml

Installing the AnalysisBase

Now we can start the installation of the Analysis Base. This can take some time. So be patient! You may need to install doxygen so before running:

sudo yum install doxygen
cd ~/ATLAS/sw/release/

rcSetupLocal -b Base,2.3.20

If at the end some packages do not install you can have a look at the log files and see what went wrong. You can clean and rebuild a single package by typing

rc clean_pkg packageName
rc compile_pkg packageName

 Hopefully its successful! :)

Installing your analysis packages

In this example I am installing the HGammaAnalysisFramework. You can follow their twiki here: https://twiki.cern.ch/twiki/bin/viewauth/AtlasProtected/HggDerivationAnalysisFramework#HGamAnalysisFramework

Go to the directory where you will be doing your analysis and setup the analysis base:

cd ~/my_xAOD_work/
rcSetupLocal Base,2.3.20

Download the packages and then run the HGam installation script to setup the environment

rc checkout_pkg atlasoff/PhysicsAnalysis/HiggsPhys/Run2/HGamma/xAOD/HGamAnalysisFramework/tags/HGamAnalysisFramework-00-02-17 

./HGamAnalysisFramework/scripts/setupRelease

This is where I download my analysis package HH2yybb:

rc checkout_pkg atlasoff/PhysicsAnalysis/HiggsPhys/Run2/HGamma/xAOD/HH2yybb/tags/HH2yybb-00-02-05

For an example of running an analysis package (Such as the HH2yybb). You can visit: https://twiki.cern.ch/twiki/bin/view/AtlasProtected/HGam_run2_ggbb#yybb_Analyis_Framework

Installing an IDE

Its a really good idea to use anIDE. This is 2015 stop using vi, nano and emacs. IntelliSense allows you to see a list of functions and their parameters while you are typing. This will save you LOADS of time so try use it!

I am using CodeBlocks in my VM. Its quite decent and has the functionality I need. 

In your main directory. such as my_xAOD_work/

rc checkout_pkg atlas-krasznaa/AODUpgrade/CMakeGenerator/trunk
mkdir IDE
cd IDE/
../CMakeGenerator/generateRootCoreCMakeProject.py

Now you have created the make file we can generate a codeblocks project by typing:

cmake -G "CodeBlocks - Unix Makefiles" .

Now open codeblocks and open the project called RootCore.cbp

You should have a working IDE now :)

Done!

Happy analysising :P

Wits High Energy Physics students honoured with prizes at the South African Institute of Physics Conference

Four students of the Wits High Energy Physics group have been honored with prizes by the South African Institute of Physics:

Stefan von Buddenbrock and Mitchell Cox, first prizes for MSc oral presentations at the Nuclear, Particle and Radiation track for




"Dark matter production in association with Higgs bosons through heavy scalar resonance at the LHC"

and 

"Online energy reconstruction on ARM for the ATLAS TileCal sROD co-processing unit"

Daniel Ohene-Kwofie, second prize for poster presentation at the Nuclear, Particle and Radiation track for

"Efficient processing of physics quantities for the Processing Unit for the upgrade of the Tile Calorimeter of ATLAS"

Chad Pelwan, first prize for poster presentation at the Condense Matter track for

“A density functional theory and magnetic resonance studies of radiation damage in plastic scintillators”

All of these students are a part of the SA-CERN programme.

Congratulations to Stefan, Mitch, Daniel and Chad!

Few pictures of the happy awardees can be seen below:

From left to right: Daniel, Stefan, Mitch and Chad.

From left to right, Deepak, Stefan, Chad, Mitch and Daniel.

Wits High Energy Physics group attending the South African Institute of Physics Conference

The South African Institute of Physics Conference is taking place this year in Port Elizabeth

http://www.saipconference.co.za

The Program is available at 

http://www.saipconference.co.za/images/Programme/Conference%20Timetable%203%20June.pdf

Below is a picture of members of the Wits High Energy Physics group who are giving talks or posters presentations. At the centre is the Head of the School of Physics, Prof. John Carter

New ADC FMC board for PROMETEO manufactured in South Africa

Figure 1: Two fully assembled ADC FMC trigger boards for PROMETEO test-bench: Made in South Africa..

An improved version of the ADC FMC trigger board to be used in the PROMETEO test-bench has been in development at Wits University. PROMETEO is the next generation test-bench system that will be used for the full certification of the upgraded front-end electronics of the ATLAS's Tile Calorimeter at CERN.

The board was developed to filter and digitise analog trigger signals coming from the front-end electronics of the TileCal. It makes use of two ADC571 chips to sample 16 differential analog signals at 40 MHz using a 12-bit resolution.  A Xilinx VC707 dev-board with a Virtex-7 FPGA is used for the control and data read-out of the ADC board. The 16 data channels to the FPGA have a combined data rate of 7680 Mbps.

Two fully assembled ADC FMC trigger boards were delivered to Wits yesterday. Oscar Kureba and Matthew Spoor are currently in the process of testing the functionality of the boards. Matthew will be off to CERN next week, where integration of the board with the rest of the PROMETEO components will take place.

8GHz Tektronix Oscilloscope Delivered

We were extremely fortunate to be granted a major funding application from the university last year for a very high end, Tektronix 8 GHz, 25 GSa/s oscilloscope. We also received an 8 GHz differential probe for the oscilloscope enabling us to get the most out of it.

We plan to use this oscilloscope for measuring PCI-Express and other high speed signals.

We don't have any photos of signal measurements yet, but the scope works perfectly so far! Check out the photos below of Oscar just after installation, and Marc when he realized that the oscilloscope runs Windows 7...

Hopefully other staff and post-grads at the university will find this oscilloscope useful to their research as well. We hope to get many years of good service from this awesome piece of equipment.

The Department of Science and Technology dealing with the Big Data problem for Sciences, the DIRISA initiative

A bunch of experts from all over the country were invited by the Department of Science and Technology (DST) via the Data Intensive Research Initiative of South Africa (DIRISA).  The event took place at the CSIR International Convention Centre on May 6th.

DIRISA is intended to connect the data intensive scientific community with the thriving cyber-infrastructure in the country. This entails many aspects of data handling and movement across infrastructure within the country and beyond. This events follows the recent signing by the DST via the Centre of High Performance Computing (CHPC) of a memorandum of understanding with the Worldwide LHC Computing Grid (WLCG). The CHPC has officially become a Tier 2. See now South Africa on the map of WLCG sites

http://wlcg.web.cern.ch

The CHPC is a Tier 1 with regards to South Africa. Around this Tier 1, Tier 2 and Tier 3 centres will be formed. DIRISA will play a pivotal role in this process.

Below is the program of the event:

Prof. Loyiso Nongxa, former Vice Chancellor and Principal of Wits, Prof. Diane Wilcox and myself were present at the event from Wits.

It was a very useful exchange of ideas and feedback on different aspects of the need of researches in terms of Big Data needs.


Below is the Dr. Daniel Adams, Director at the DST giving one of the overview talks.

Below is Prof. Colin Wright, known to many ay Wits, giving one of the introductory talks.

Below a couple of pictures of the invited delegates

Conversing with Prof. Colin Wright during one of the breaks.

Splited into groups. Here is the group from sciences.

Conversing with Dr. Happy Sithole, the director of the CHPC. Prof. Loyiso Nongxa at the end of the table.

Visiting Parsec, a company considered for the production of the ATLAS TileCal's sROD module

Few of us from Wits, Bheku Zulu, Ela Romanonowska, from the Wits Enterprise, Thomas Wrigley and myself visited Parsec

http://www.parsec.co.za

Parsec is a high-tech company that produces advanced electronics for the domestic and foreign markets. We are considering using Parsec for the population of the boards of the sROD (super Read Out Driver) for the upgrade of the off-detector electronics of the Tile Calorimeter. Parsec just moved to a new building about 15k from Pretoria.

Below few pictures of the visit