This post describes one way (the one I often use) and little helper tools to build the vector engine (VE) operating system VEOS, the kernel modules ve_drv and vp, libved, veoffload, veoffload-veorun. The officially released repositories for those components can be found at https://github.com/veos-sxarr-NEC, do clone or fork them and experiment with changes, if you’d like to improve or contribute to their development!
VEOS recently (since version 1.3) started supporting Debian packages, but the default installation and way of distributing and updating it is still by using RPM packages on top of RHEL 7.X or CentOS 7.X. Therefore all instructions in this post and all tools are geared towards the RPMs based setups. Some day this might be extended towards Debian… Contributions are very welcome.
Store The Configuration
Before doing changes to your system, store the configuration. Since the changed packages will replace previously installed RPMs, save the list of installed RPMs such that you can determine later to which version you want to “downgrade” the packages when you want to switch back to the original, unmodified software. Either save RPM versions selectively or all at once, eg. with
rpm -qa | sort >rpmlist.orig
The build-veos Repository
The little helpers for building packages from cloned or modified repositories are located in the github repository https://github.com/efocht/build-veos. Clone this repository locally:
git clone https://github.com/efocht/build-veos.git
The helper scripts are located in the subdirectory
The scripts named
x/bld.* can be used for building RPMs from their
repositories. They should be invoked from within the top level
directory of the repository. The environment variable RELEASE can be
set to reflect the special release version you are building. The
environment variable VERSION should reflect the version of the
package normally set in its configure.ac file at the top level. When
the freshly created packages should immediately replace the installed
RPMs on the build system, set the environment variable:
x/install_prereqs is a pretty dumb collection of
prerequisites needed on the build system. Run it with a
it invokes yum and needs root permissions.
x/vemods_load will, respectively,
stop VEOS services and unload the kernel modules ve_drv and vp, or
load them and start the VEOS services. Unloading the modules is needed
in order to be able to update the modules RPMs and the VEOS RPMs.
NOTE: the module unload/load scripts ignore ve_peermem module and the vemmd service. Deal with these manually, if needed.
Example: Building veoffload
Clone repositories, checkout proper branch
mkdir BLD cd BLD git clone https://github.com/efocht/build-veos.git ln -s build-veos/x . git clone https://github.com/SX-Aurora/veoffload cd veoffload # checkout multithreaded_vh branch git checkout multithreaded_vh
# from within the veoffload repository env VERSION=1.2.2e ../x/bld.veoffload
The veoffload package ignores (right now) the value of the RELEASE env variable, therefore the packages produced are:
Wrote: /root/rpmbuild/SRPMS/veoffload-1.2.2e-1.el7.centos.src.rpm Wrote: /root/rpmbuild/RPMS/x86_64/veoffload-1.2.2e-1.el7.centos.x86_64.rpm Wrote: /root/rpmbuild/RPMS/x86_64/veoffload-devel-1.2.2e-1.el7.centos.x86_64.rpm Wrote: /root/rpmbuild/RPMS/x86_64/veoffload-debuginfo-1.2.2e-1.el7.centos.x86_64.rpm
You might want (or need) to build the package veoffload-veorun,
too. Follow the same procedure and use the script
Example: Building An Experimental VEOS
Instructions for building and testing my dma-bulk-hugepage branch of veos, ve_drv, vp, libved. This code is experimental and comes with no guarantees for anything. Use on your own risk. The procedure described in this section is documented here, too, with slightly more explanations on the motivation for it. It significantly speeds up data transfers between VH and VE which use the privileged (AKA “system”) DMA descriptors and unregistered/unpinned buffers.
Clone repositories, checkout proper branch
mkdir BLD cd BLD GITH="https://github.com/efocht" for repo in build-veos veos libved ve_drv-kmod vp-kmod vhcall-memtransfer-bm; do git clone $GITH/$repo.git done ln -s build-veos/x . for repo in veos libved ve_drv-kmod vp-kmod; do cd $repo git checkout dma-bulk-hugepage cd .. done
Build RPMs and replace old ones by new ones
# stop VEOS and VE related services, unload VE modules x/vemods_unload # replace installed RPMs by the newly built ones export RPMREPLACE=1 cd libved ../x/bld.libved cd .. cd vp-kmod ../x/bld.vp cd .. cd ve_drv-kmod ../x/bld.ve_drv cd .. cd veos ../x/bld.veos cd .. # load VE modules, start VEOS and VE related services x/vemods_load