Thanks to the Wits Physics workshop who did a great job with mounting the power supply and brackets.
 |
| Final Wandboard Array installed in rack. |
 |
| Wandboard array fresh from the workshop. |
Friday, 16 May 2014
32 Core - 8 Wandboard Array Rack Mounted
We thought it was time to make our Wandboard Array a little bit more formal. We purchased a small tray for our rack and started work on mounting the array next to its power supply.
Thanks to the Wits Physics workshop who did a great job with mounting the power supply and brackets.
Thanks to the Wits Physics workshop who did a great job with mounting the power supply and brackets.
Tuesday, 13 May 2014
Extra Steps for Building a Wandboard (i.MX6Q) Image
I have built several Linaro based images for the Wandboard or Freescale i.MX6Q SoC, and it's a seemingly simple process of building the kernel, partitioning and installing the Linaro rootfs to a SD card. One hit's a wall when anything 'fancy' needs to be done on this image... this post aims to document how to finish off the image so that it can be used for kernel development and other advanced tasks.
Please note that this is a fairly advanced howto. Most of the concepts here can be found in other places online. The methodology behind the i.MX6 libraries comes from 'reverse engineering' the LTIB install scripts from Freescale, yocto, and lots of reading. I'm sure there is a better way to do this - but I don't know it (I'm interested to hear though)!
Linux Source Code and Modules
The Linux kernel source code should be cleaned (make ARCH=arm clean) and copied onto the image (or SD card - I'll use the terms randomly) into the /usr/src/linux directory.
Before you clean the tree, build the modules and headers and install them to a known path to be copied onto your SD card root filesystem.
make modules_install INSTALL_MOD_PATH=/some/directory/
make headers_install INSTALL_HDR_PATH=/some/directory/
Copy the contents of /some/directory to into the SD card /lib directory. You should now have a new directory: /lib/modules/3.0.35Linux+ or something similar. The headers_install directory goes into /usr.
When you boot with the new SD card, you will need to modify some symlinks (build and source) that reference the wrong place (they will be linking to directories from your build machine, which are obviously not valid any more).
cd /lib/modules/3.0.35Linux+/
ls -l
You should see the wrong links. Now to remove them and re-add them, correctly:
rm ./build
rm ./source
ln -s /usr/src/linux ./build
ln -s /usr/src/linux ./source
These commands will have to be run with sudo or as root.
Something you may also have to do, in case you cleaned the kernel source tree too thoroughly, or you cross compiled (the kernel scripts directory is full of x86 binaries which wont work on ARM: modpost, etc.), is ensure your proper .config file is there and run:
make ARCH=arm oldconfig
make ARCH=arm prepare
make ARCH=arm modules_prepare
I've also found that doing a module build fixed up some errors!
make ARCH=arm modules
Download the i.MX6 Libraries
Download and extract the Freescale BSP somewhere onto your PC (not the Wandboard). There is a source directory with loads of .gz files for various applications. We are interested in several of these. This section will explain how to install the important ones.
A list of the files we will be working with is below. Copy them onto the SD card before booting, or SCP them across. From what I have seen, the 3.0.35-4.1.0 and 3.10.17-1.0.0 are basically the same. The 3.10.17 files can be acquired from a partial yocto installation - Google can probably help with finding the files otherwise!
imx-lib-*.tar.gz
imx-vpu-*.bin
firmware-imx-*.bin
imx-test-*.tar.gz
gpu-viv-bin-mx6q-*hfp.bin (the version from the Freescale BSP seems to be soft-fp... it won't work.)
gpu-viv-g2d-*.bin
fsl-gpu-sdk-*.bin
Some of these are bins, with things to agree to before they extract. Go ahead and extract everything in preparation for the installation steps.
Install the i.MX6 Libraries
First, a few exports to make our lives easier:
export KERNELDIR='/usr/src/linux'
export INCLUDES='-I$KERNELDIR/include -I$KERNELDIR/drivers/mxc/security/rng/include -I$KERNELDIR/drivers/mxc/security/sahara2/include'
As per the usual, make sure the KERNELDIR variable points to your specific kernel directory. Note that if you have installed the kernel headers, etc. properly in the steps above, you may not need to do these exports and you can leave the INCLUDE= part out of the make commands below.
firmware:
Simply copy the contents of the firmware-imx* directory into /lib so that you have new files in /lib/firmware/vpu, etc.
imx-lib:
From the imx-lib directory:
make -j1 PLATFORM="IMX6Q" INCLUDE="$INCLUDES"
sudo make PLATFORM="IMX6Q" install
If all compiled and copied, you should now see a bunch of new libraries in /usr/lib! Congratulations!
imx-vpu:
Even if you don't want to use the VPU, this is a dependency for the imx-tests. From the imx-vpu directory:
make -j1 PLATFORM="IMX6Q" INCLUDE="$INCLUDES"
sudo make PLATFORM="IMX6Q" install
If all compiled and copied, you should now see a bunch of new libraries in /usr/lib! Congratulations!
gpu-viv-bin and gpu-viv-g2d:
Copy the contents of the archives into your root. /opt and /usr will now contain new files.
imx-test:
From the imx-test directory:
make -j1 PLATFORM="IMX6Q" test
At this point you can run some of the unit tests that compiled successfully (not all will have) from the ./platform/IMX6Q/autorun*.sh files.
sudo make PLATFORM="IMX6Q" install
If all compiled and copied, you should now see a bunch of new libraries in /usr/lib! Congratulations!
gpu-viv-bin:
The GPU drivers and binaries are closed source, so it's a matter of extracting the files into the correct place. Search for gpu-viv-*.gz and copy it onto your Wandboard.
Boot, extract it and cd into the new directory. You will see an 'opt' and a 'usr' directory. Run the following commands:
sudo cp -Rv ./opt/* /opt/
sudo cp -Rv ./usr/* /usr/
Make sure the files copied into the correct places.
Please note that this is a fairly advanced howto. Most of the concepts here can be found in other places online. The methodology behind the i.MX6 libraries comes from 'reverse engineering' the LTIB install scripts from Freescale, yocto, and lots of reading. I'm sure there is a better way to do this - but I don't know it (I'm interested to hear though)!
Linux Source Code and Modules
The Linux kernel source code should be cleaned (make ARCH=arm clean) and copied onto the image (or SD card - I'll use the terms randomly) into the /usr/src/linux directory.
Before you clean the tree, build the modules and headers and install them to a known path to be copied onto your SD card root filesystem.
make modules_install INSTALL_MOD_PATH=/some/directory/
make headers_install INSTALL_HDR_PATH=/some/directory/
Copy the contents of /some/directory to into the SD card /lib directory. You should now have a new directory: /lib/modules/3.0.35Linux+ or something similar. The headers_install directory goes into /usr.
When you boot with the new SD card, you will need to modify some symlinks (build and source) that reference the wrong place (they will be linking to directories from your build machine, which are obviously not valid any more).
cd /lib/modules/3.0.35Linux+/
ls -l
You should see the wrong links. Now to remove them and re-add them, correctly:
rm ./build
rm ./source
ln -s /usr/src/linux ./build
ln -s /usr/src/linux ./source
These commands will have to be run with sudo or as root.
Something you may also have to do, in case you cleaned the kernel source tree too thoroughly, or you cross compiled (the kernel scripts directory is full of x86 binaries which wont work on ARM: modpost, etc.), is ensure your proper .config file is there and run:
make ARCH=arm oldconfig
make ARCH=arm prepare
make ARCH=arm modules_prepare
I've also found that doing a module build fixed up some errors!
make ARCH=arm modules
Download the i.MX6 Libraries
Download and extract the Freescale BSP somewhere onto your PC (not the Wandboard). There is a source directory with loads of .gz files for various applications. We are interested in several of these. This section will explain how to install the important ones.
A list of the files we will be working with is below. Copy them onto the SD card before booting, or SCP them across. From what I have seen, the 3.0.35-4.1.0 and 3.10.17-1.0.0 are basically the same. The 3.10.17 files can be acquired from a partial yocto installation - Google can probably help with finding the files otherwise!
imx-lib-*.tar.gz
imx-vpu-*.bin
firmware-imx-*.bin
imx-test-*.tar.gz
gpu-viv-bin-mx6q-*hfp.bin (the version from the Freescale BSP seems to be soft-fp... it won't work.)
gpu-viv-g2d-*.bin
fsl-gpu-sdk-*.bin
Some of these are bins, with things to agree to before they extract. Go ahead and extract everything in preparation for the installation steps.
Install the i.MX6 Libraries
First, a few exports to make our lives easier:
export KERNELDIR='/usr/src/linux'
export INCLUDES='-I$KERNELDIR/include -I$KERNELDIR/drivers/mxc/security/rng/include -I$KERNELDIR/drivers/mxc/security/sahara2/include'
As per the usual, make sure the KERNELDIR variable points to your specific kernel directory. Note that if you have installed the kernel headers, etc. properly in the steps above, you may not need to do these exports and you can leave the INCLUDE= part out of the make commands below.
firmware:
Simply copy the contents of the firmware-imx* directory into /lib so that you have new files in /lib/firmware/vpu, etc.
imx-lib:
From the imx-lib directory:
make -j1 PLATFORM="IMX6Q" INCLUDE="$INCLUDES"
If all compiled and copied, you should now see a bunch of new libraries in /usr/lib! Congratulations!
imx-vpu:
Even if you don't want to use the VPU, this is a dependency for the imx-tests. From the imx-vpu directory:
make -j1 PLATFORM="IMX6Q" INCLUDE="$INCLUDES"
If all compiled and copied, you should now see a bunch of new libraries in /usr/lib! Congratulations!
gpu-viv-bin and gpu-viv-g2d:
Copy the contents of the archives into your root. /opt and /usr will now contain new files.
imx-test:
From the imx-test directory:
make -j1 PLATFORM="IMX6Q" test
At this point you can run some of the unit tests that compiled successfully (not all will have) from the ./platform/IMX6Q/autorun*.sh files.
If all compiled and copied, you should now see a bunch of new libraries in /usr/lib! Congratulations!
gpu-viv-bin:
The GPU drivers and binaries are closed source, so it's a matter of extracting the files into the correct place. Search for gpu-viv-*.gz and copy it onto your Wandboard.
Boot, extract it and cd into the new directory. You will see an 'opt' and a 'usr' directory. Run the following commands:
sudo cp -Rv ./opt/* /opt/
sudo cp -Rv ./usr/* /usr/
Make sure the files copied into the correct places.
Tuesday, 6 May 2014
Upgraded Wandboard Cluster
We have invested in a few more Wandboards to upgrade our mini-cluster. Since I repurposed two of the existing boards for my PCI-Express research, we now have 8 Wandboards in the cluster which makes 32 cores! A little more power for scientific applications on ARM...
Besides having more processing power available, I have made some enhancements to the power distribution board. The thin purple, pink and red wires in the background will be connected to one of the Wandboard I2C inputs to enable easy and accurate voltage, current and power measurements of the DC supply to the cluster. The excellent TI INA219 chip is used for this. It's possible to read these values from the Linux command-line so we can use a shell script to plot power values directly to a file!
Besides having more processing power available, I have made some enhancements to the power distribution board. The thin purple, pink and red wires in the background will be connected to one of the Wandboard I2C inputs to enable easy and accurate voltage, current and power measurements of the DC supply to the cluster. The excellent TI INA219 chip is used for this. It's possible to read these values from the Linux command-line so we can use a shell script to plot power values directly to a file!
Subscribe to:
Posts (Atom)