Tuesday, 12 May 2015

8GHz Tektronix Oscilloscope Delivered

We were extremely fortunate to be granted a major funding application from the university last year for a very high end, Tektronix 8 GHz, 25 GSa/s oscilloscope. We also received an 8 GHz differential probe for the oscilloscope enabling us to get the most out of it.

We plan to use this oscilloscope for measuring PCI-Express and other high speed signals.

We don't have any photos of signal measurements yet, but the scope works perfectly so far! Check out the photos below of Oscar just after installation, and Marc when he realized that the oscilloscope runs Windows 7...

Hopefully other staff and post-grads at the university will find this oscilloscope useful to their research as well. We hope to get many years of good service from this awesome piece of equipment.


Monday, 11 May 2015

The Department of Science and Technology dealing with the Big Data problem for Sciences, the DIRISA initiative

A bunch of experts from all over the country were invited by the Department of Science and Technology (DST) via the Data Intensive Research Initiative of South Africa (DIRISA).  The event took place at the CSIR International Convention Centre on May 6th.

DIRISA is intended to connect the data intensive scientific community with the thriving cyber-infrastructure in the country. This entails many aspects of data handling and movement across infrastructure within the country and beyond. This events follows the recent signing by the DST via the Centre of High Performance Computing (CHPC) of a memorandum of understanding with the Worldwide LHC Computing Grid (WLCG). The CHPC has officially become a Tier 2. See now South Africa on the map of WLCG sites

http://wlcg.web.cern.ch

The CHPC is a Tier 1 with regards to South Africa. Around this Tier 1, Tier 2 and Tier 3 centres will be formed. DIRISA will play a pivotal role in this process.

Below is the program of the event:




Prof. Loyiso Nongxa, former Vice Chancellor and Principal of Wits, Prof. Diane Wilcox and myself were present at the event from Wits.

It was a very useful exchange of ideas and feedback on different aspects of the need of researches in terms of Big Data needs.


Below is the Dr. Daniel Adams, Director at the DST giving one of the overview talks.


Below is Prof. Colin Wright, known to many ay Wits, giving one of the introductory talks.


Below a couple of pictures of the invited delegates



Conversing with Prof. Colin Wright during one of the breaks.


Splited into groups. Here is the group from sciences.


Conversing with Dr. Happy Sithole, the director of the CHPC. Prof. Loyiso Nongxa at the end of the table.



Thursday, 7 May 2015

Visiting Parsec, a company considered for the production of the ATLAS TileCal's sROD module

Few of us from Wits, Bheku Zulu, Ela Romanonowska, from the Wits Enterprise, Thomas Wrigley and myself visited Parsec

http://www.parsec.co.za

Parsec is a high-tech company that produces advanced electronics for the domestic and foreign markets. We are considering using Parsec for the population of the boards of the sROD (super Read Out Driver) for the upgrade of the off-detector electronics of the Tile Calorimeter. Parsec just moved to a new building about 15k from Pretoria.

Below few pictures of the visit






Thursday, 30 April 2015

UCT Workshop: High Performance Computing with CUDA 2015


Last week I attended the third annual GPGPU conference at the University of Cape Town.  Speakers included two international NVIDIA fellows, John Stone and Manuel Ujaldón from the Universities of Malaga and Illinois, as well as local speaker Bruce Merry who is working on GPU parallel processing with the Square Kilometer Array. Relatively new to the world of GPGPU myself, I was exposed to a privileged amount of expert knowledge. John, Miguel and Bruce represent a well of insight into the field of parallel processing - both technical aspects as well as a familiarity with the politics of GPUs and the microprocessor industry as a whole. 

There was a theme throughout the lectures of achieving processor performance excellence through a close relationship between programmer and hardware. Learning to harness the power of potentially millions of independent processing units requires attention to memory management and inter-processor relationships. GPUs have several types of memory, varying by size and speed of access and how the many processing units inside share all that memory must be carefully planned. To get the best possible results, one needs to ensure that as many processing units as possible are active at any given time. However, as John pointed out more than once, one cannot measure performance in GFLOPs only, as this metric does not necessarily represent 'scientific throughput' in a simple way. Perhaps the most valuable aspect of the workshop was being privy to John and Bruce's code and coding thought processes. John has done some interesting work on taking quantum-chemical graphic simulation generation from the realm of static image generation to real-time interactive animations. John and Manuel have been working with GPUs since GPGPU's early days, and as such talked us through both the future and the past of GPGPU. Parallel processing does not follow the CPU pattern of ever faster clock rate and more advanced control architecture, but rather seeks a balance between parallel abstractions, memory management, and speed. A crucial constraint in all high-throughput computing environments, like ATLAS, is I/O throughput. Future architectures, transfer protocols (like NVLink),and processor cluster configurations aim to improve this parameter - they themselves are a function of technology as well as the cooperation of big businesses who do not necessarily benefit from cooperating with competitors. 

This information is being put to work in my research into the role GPGPU will play in the high-throughput TileCal detector. The GPU(s) I will use will be working in tandem with ARM processors, which amongst other benefits, have an open-source architecture allowing for more versatile and customised designs. If you're interested in GPGPU techniques and their role in ATLAS, keep an eye on this blog for more parallel processing posts.

Wednesday, 29 April 2015

Higgs to di-photon workshop hosted by ATLAS

CERN has recently hosted an HGamma workshop at the Business Center in St-Genis. This 2.5 day workshop (from Monday April 27th to Wednesday April 29th) was intended to assess the readiness of HGamma analyses for run2. HGamma is the name given to this particular decay channel being studied. A walkthrough for early analyses was given followed by a review of the status of combined performance and common tools. This was followed by a look at all longer term analyses which require more data.

The Wits members involved in the HGamma group include:
  1. Luis March 
  2. Mthokozisi Masuku 
  3. Bruce Mellado 
  4. Stefan von Buddenbrock 
  5. Deepak Kar 
  6. Oscar Kureba 
  7. Shell-May Liao 
  8. Robert Graham Reed 
  9. Xifeng Ruan 
  10. Kehinde Gbenga Tomiwa

The University of the Witwatersrand had a total of five presentations. The presenters and topic overviews were:

  • Luis March - This talk discussed the new Missing Transverse Momentum (ETmiss) definition that will combat the issues associated with the increase in collision energy. The presentation showed the status and development of the ETmiss for Run-II, comparing the performance of two ET miss definitions by using Monte Carlo datasets. In addition, the status of an ETmiss tool which will be used by the missing energy groups was shown.
  • Robert Reed - An overview talk about the status of the analysis framework for the Higgs to di-photon and di-bjet decay channel. This is a sub analysis of the HGamma channel. The talk included a short tutorial on how to run the analysis and the selection criteria involved. Further studies were presented regarding jet energy resolutions as well as simulation type comparisons.

  • Xifeng Ruan - Presented three talks:
  1. This talk presented the inauguration of a new study involving the Higgs and missing transverse energy (ETmiss). This is a primary candidate in the study for beyond the Standard Model. The talk involved the understanding of the missing transverse energy from the 2011, 2012 and 2015 data to look for new physics. 
  2. A summary of the recent simulation samples was shown. The simulation samples are the theoretical estimates we should observe during data taking. The ETmiss group has requested new samples for study for the new data period in 2015. 
  3. A detailed look at the future prospects of the ETmiss study was discussed. This included the intermediate and high ETmiss/BSM groups. This also included all the ongoing efforts in the group.

Tuesday, 14 April 2015

Radio Interview with the Wits-ATLAS group

Thomas Wrigley and myself were interviewed by POWERFM98.7 Radio station in Johannesburg. Great host, Lawrence Tlhabane and a nice experience. The recording of the interview is available at:

https://soundcloud.com/powerfm987/science-why-dark-matters

More about the radio station at.

http://www.powerfm.co.za


Friday, 10 April 2015

New ADC daughter board for the ATLAS TileCal PROMETEO test-bench

An improved version of the ADC daughter board used in the PROMETEO test-bench has been in development at WITS. PROMETEO is the new test-bench system that will be used for the full certification of the upgraded front-end electronics of the ATLAS TileCal.

The board was developed to filter and digitise analog trigger signals coming from the front-end electronics of the TileCal. It makes use of two ADC571 chips to sample 16 differential analog signals at 40 MHz using a 12-bit resolution.  A Xilinx VC707 dev-board with a Virtex-7 FPGA is used for the control and data read-out of the ADC board. The 16 data channels to the FPGA have a combined data rate of 7680 Mbps.

This week we received the newly manufactured PCBs which have since been sent off for assembly. Once final testing is completed the new board will be fully integrated into the ATLAS system.


Wednesday, 8 April 2015

Press release by Wits regarding the re-start of the LHC and the role of Wits scientists

New discoveries await scientists

8 April 2015

Photo:CERN

As the world’s most powerful particle accelerator, the Large Hadron Collider (LHC) at the European Organization of Nuclear Research (CERN) in Geneva, Switzerland, re-started over the weekend physicists from the University of the Witwatersrand in Johannesburg are ready to further explore the frontiers of science that led to the discovery of the Higgs boson particle in 2012.
The circulation beams rebooted on Sunday morning, 5 April 2015 – a major milestone towards resuming proton-to-proton collisions and delivering protons at a new energy frontier before the summer in the northern hemisphere. "The return of beams to the LHC rewards a lot of intense, hard work from many teams of people," said Head of CERN’s Beam Department, Paul Collier. "It is very satisfying for our operators to be back in the driver’s seat, with what is effectively a new accelerator to bring on-stream, carefully, step-by-step.”
"Operating accelerators for the benefit of the physics community is what CERN is here for,” said CERN Director General, Professor Rolf-Dieter Heuer. 
The team reacts as the Large Hadron Collider restarts. Image: CERNThe historic observation three years ago of the Higgs boson, a puzzle piece in the Standard Model of Particle Physics, opened up possibilities of more exotic particles waiting to be discovered.
“There are many questions left unanswered in the Universe, some of which we hope to answer when the Large Hadron Collider (LHC) is once again fully operational,” says Professor Bruce Mellado from the Wits School of Physics.
An expert on the Higgs boson, Mellado – who leads the University’s involvement in the Atlas experiment at CERN - was recently appointed co-convener in the Higgs Cross-section Group at CERN that puts together the work of more than 100 theorists and experimentalists, and oversees seven subgroups that are setting up standards for the study of the properties of the Higgs boson with the data that the Large Hadron Collider will provide. Mellado has also been made a member of the Large Hadron Electron Collider (LHeC) Coordination Group – a new proposed R12-billion facility to be constructed.

Wits and the ATLAS experiment
Mellado and colleagues in the High Energy Physics (HEP) group in the Wits School of Physics are investigating peculiarities seen in the Higgs boson results made publicly available by the ATLAS experiment.
The possible explanations to these peculiarities range from a set of random statistical fluctuations, which may disappear with more data, to a much more interesting possibility of the observation in laboratory conditions of dark matter particles. This would signify a finding as important as that of the Higgs boson.
HEP will be quite active in this investigation and crucial support from the Department of Science and Technology, through the SA-CERN programme and bursaries, as well as University contributions will play a critical role in this endeavour. Professor Alan Cornell, a high-energy particle physics theorist from the Mandelstam Institute for Theoretical Physics at Wits, collaborates with experimentalists of the HEP group to devise new scenarios unexplored before. Another new member to the HEP group, Dr Deepak Kar from the University of Glasgow, will bring a complimentary dimension to the group through his knowledge and experience.
Since the start of 2013, the Wits-ATLAS group has submitted and/or published over 60 proceedings and research papers. Students Harshna Jivan, Mitchell Cox and Robert Reed have received five prizes in the same period of time. The group has been honoured with the First Time Inventor Award (2014) and the Most Cited Researcher Award (2015) by the University.

Wits played an active role during shutdown
The HEP group was very active during the shutdown of the LHC since 2013 – in order to prepare the collider for almost double the previous collision energies. Members have designed and locally produced High Voltage, LED and ADC electronic boards that are used in a mobile integrity-checking system that validates and consolidates the electronics on the ATLAS detector at the LHC. These boards along with custom software, also developed by the group, used a new design, called Prometeo, which is intended for the upgrade of the LHC expected in the year 2022.

Dealing with data explosion and development of low-cost computers for South Africa
Storing the large amount of data – in the order of a few Petabytes per second – that the LHC is expected to generate in the near future, is a major concern to physicists. In response, the HEP group also initiated the development of software tools that integrate the new hardware into the ATLAS detector control system, and the group is developing a novel way to deal with the data explosion by processing the data, in real time, using energy efficient and cost effective processors found in mobile phone devices.
These high-end electronics are being designed and developed in the new High-throughput Electronics Laboratory (HTEL) in the Wits School of Physics. The lab was inaugurated in November last year by the Wits Vice-chancellor and Principal, Professor Adam Habib, the Dean of the Faculty of Science, Professor Helder Marques, and Head of the School of Physics, Professor John Carter.
 As a spinoff of this high-end electronics research, the HTEL also houses the design of low-cost computers for schools in Gauteng. The Technology Innovation Agency has awarded the HTEL with a grant to launch a pilot project at selected schools to study and develop these devices to best suit the education environment

Wits’ efforts with ALICE
The re-start of the LHC at the new energy frontier and with greater intensity also brings reinvigorated excitement for the ALICE experiment at the LHC. Professor Zeblon Vilakazi, Deputy Vice-Chancellor: Research, leads Wits’ efforts at the ALICE experiment. The ALICE experiment seeks to investigate the state of matter that excited a millionth of a second after the Big Bang - known as the quark gluon plasma — a soup of matter particles that cooled down to form atomic nuclei that we observe today.

Synergies with nuclear physics and material sciences
Professor Elias Sideras-Haddad leads synergistic activities between high-energy physics and the HTEL with nuclear physics and material sciences. At the forefront of these investigations lies a unique set of South African accelerator-based facilities that allow scientists to reproduce the challenging conditions of radiation at the LHC. In particular, radiation damage of ATLAS detector components is studied using several techniques developed for the material sciences.

http://www.wits.ac.za/newsroom/newsitems/201504/26074/news_item_26074.html


Thursday, 26 March 2015

Building ATLAS 3.11 on ARM with gcc 4.9

I am in the process of re-running our HPL benchmarks on the ARM boards. The latest version of ATLAS, 3.11.32 has several optimisations for ARM and specifically ARMv8 which I would like to compare to the older and more stable ATLAS 3.10. I also discovered that gcc 4.9.2 makes quite a big difference in the CoreMark benchmark on all of the platforms we have available.

This post is just some quick documentation for myself as well as others as to how to get ATLAS to compile on the Wandboard, Cubieboard A20 and the Applied Micro X-Gene (Mustang).

In all cases I found that the recipe for success is to manually specify the compilers. Of course, setting the CPU frequency governor to performance is also required.

The Tegra-K1 compile doesn't seem to be working yet. I'll post as soon as I'm successful!

On all of the boards the following commands work to set the CPU governors:
sudo su
echo "performance" > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor

and on the Tegra-K1 it's required to turn off the power saver core:
echo 0 > /sys/devices/system/cpu/cpu0/cpuquiet/...

When you are done setting up the CPU governors, find out the running frequency and exit the su:
cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq
exit

Go ahead and create a directory where you would like to download and build ATLAS to. Inside that directory, fetch the latest unstable version of atlas and if required, rename it.
wget http://sourceforge.net/projects/math-atlas/files/Developer%20%28unstable%29/3.11.32/atlas3.11.32.tar.bz2/download

Extract it and head to a build directory:
tar xf ./atlas3.11.32.tar.bz2
cd ATLAS
mkdir build
cd build

Now the magic happens... The configure command has loads of options but I find that in the unstable (3.11) range, the defaults work well for ARM. It just needs a little help. We have to manually specify the CPU frequency and since I am using gcc-4.9 and gfortran-4.9 I force them. I found that even if you tell the system that gcc and gfortran are the respective -4.9 versions, it doesnt always work.

Go ahead and type the following command to do the configure:
../configure -m <fpu_freq> -C acg gcc-4.9 -C if gfortran-4.9 -D c -DWALL

Everything should work out with no errors. If there are, then you need to delete the contents of the build directory and start again. Try different configure flags.

When the configure is done with no errors simply run
make build
and give it an hour or two!

Tuesday, 3 March 2015

Interview with the SABC

This week our HEP group had a great visit from the SABC. Their crew were filming a piece for the show “Teenagers On a Mission” (Tomz). They were very interested in the 3D printing development that we have been doing for the Mini PC project. We were able to outline the whole 3D printing process and give them quick demonstration using our 3D scanning and printing setup in the lab.


Thomas Wrigley giving his take on 3D printing in South Africa


Tomz, part factual, part fictional with some drama, comedy and science fiction will be broadcast in a fair spread of Xhosa, Sotho, Sepedi, English and Zulu with full subtitles in English”


 Airs every week: Monday 4:00-4:30pm, Tuesday 4:00-4:30pm on SABC 1