NeuroDebian Insider

Running 64bit Matlab on 32bit host OS

team@neuro.debian.net (NeuroDebian team) — Mon, 03 Jun 2013 14:00:00 -0000

Running 64bit Matlab on 32bit host OS

Some of you have experienced problems due the recent move of Mathworks to drop 32-bit Linux builds of their products (i.e. Matlab R2013a and co.). Please note that this is not the first time Mathworks values its own costs higher than the benefits of a few scientists. In 1998 PowerPC builds for Macs were abandoned, causing a furious reaction of the community.

Luckily, users of the fresh Debian stable release wheezy (or more recent variants of Debian and its derivatives) who still need a 32bit OS on 64bit-capable hardware can take advantage of the new multiarch support. Multiarch allows for multiple architecturesi to co-exist on a hardware/kernel that is capable of supporting both (e.g. i386 and amd64).

Below we describe how you can use multiarch support and in few simple steps that prepare your existing 32bit user-land for running 64bit Matlab.

Procedure

[2-10 min] Install 64-bit kernel and reboot:
```
sudo apt-get install linux-image-amd64
```
[1-3 min] Enable multi-arch support for amd64 architecture:
```
sudo dpkg --add-architecture amd64
sudo apt-get update
```

[1-5 min] Install 64bit libraries (and compilers) needed for matlab:

sudo apt-get install libstdc++6:amd64 zlib1g:amd64 libncurses5:amd64 \
  libxp6:amd64 libstdc++6-4.4-dev:amd64 libxt6:amd64 libxmu6:amd64 libxtst6:amd64 \
  g++:amd64 gcc:amd64 binutils:amd64

Now your 64bit matlab (which you hopefully “registered” with matlab-support) is ready to run.

If you have any comments (typos, improvements, etc) – feel welcome to leave a comment below, or contact us .

NeuroDebian nd* tools

team@neuro.debian.net (NeuroDebian team) — Sat, 14 Apr 2012 22:16:00 -0000

NeuroDebian nd* tools

One of the goals of NeuroDebian is to provide recent versions of scientific software on stable Debian (and Ubuntu) deployments. That is why we build (whenever possible) every new package not only for the Debian unstable (the entry point of packages into Debian) but also for Debian testing and stable, and Ubuntu releases. To automate such procedure we prepared few rudimentary wrappers around cowbuilder allowing to build packages in isolated environment. Also we provide a backport-dsc script to ease backporting with optional application of per-release patchsets. In this blog post we would like to introduce you to these tools. They will be of use for anyone working on a package intended to be uploaded to NeuroDebian repository or anyone interested to verify if package could be easily backported. With a single command you will be able to build a given Debian source package across distributions. As a result you will verify that there are no outstanding backport-ability issues or compatibility problems with core components (e.g. supported versions of Python) if your source package excercises test suites at build time.

Procedure

[1-20 min] If you are not running Debian-based distribution, Install NeuroDebian VM; otherwise just add apt sources for NeuroDebian repository.
[<1 min] Install the neurodebian-dev package providing nd* tools:
```
sudo apt-get install neurodebian-dev
```
[1-5 min] Adjust default configuration (sudo vim /etc/neurodebian/cmdsettings.sh) used by nd commands to
- point cowbuilderroot variable to some directory under brain account, e.g. ~brain/debs (should be created by you)
- remove undesired releases (e.g. deprecated karmic) from allnddist and alldist
- adjust mirror entries to use the Debian mirror and Ubuntu mirror of your choice or may be even point to your approx apt-caching server

[10-60 min] Create the COWs for all releases you left in the configuration file:
```
sudo nd_adddistall
```

Building

At this point you should be all set to build packages for all distributions with a single command. E.g.:

sudo nd_build4all blah_1-1.dsc

should take the .dsc file you provide, and build it for main Debian sid and all ND-supported releases of Debian and Ubuntu. nd_build4allnd would build only for the later omitting the vanilla Debian sid. The highlevel summary either builds succeed or failed get reported in summary.log in the same directory, pointing to .build log files for the corresponding architecture/release.

Troubleshooting Failing Build

Provide --hookdir cmdline pbuilder argument to point to a hook which would get kicked in by pbuilder upon failure, e.g.:

sudo apt-get install git
git clone https://github.com/neurodebian/neurodebian
sudo nd_build4debianmain *.dsc -- --hookdir $PWD/neurodebian/tools/hooks

If you have any comments (typos, improvements, etc) – feel welcome to leave a comment below, or contact us .

Chroot workaround for fslview (HOWTO)

team@neuro.debian.net (NeuroDebian team) — Mon, 12 Dec 2011 12:05:00 -0000

Chroot workaround for fslview (HOWTO)

Note

With the release of FSLView 4.0.0b this workaround should no longer be necessary. However, the technology is stil equally useful to work around similar problems with other software.

Preamble

Sometimes research software lags behind developments in libraries it relies upon, because necessary changes to the code might require considerable effort, and thus time. That leads to difficulties building those tools using up-to-date library versions due API incompatibility.

That is what happened with fslview from the FSL suite. Today, at the end of 2011, it still relies on Qt3 and VTK GUI support for it.

Note

Last version of Qt3, 3.3.8, was released in February 2007, with official support by Trolltech terminated later that year.

Qt4 was first released in 2005, and current stable series 4.7 released appeared in September 2010.

Because of its age and discontinued upstream support, Qt3 was orphaned in Debian, and tools relying on it were encouraged to migrate to use Qt4. As a result, although Qt3 itself is still present in Debian (and thus Ubuntu), VTK GUI support for Qt3 (package libvtk5.4-qt3) which fslview uses, was removed from Debian due to Qt3 deprecation. It should be made clear that it was not removed to annoy people, but rather because it became unfeasible to maintain its robust building and functioning. So nowadays fslview is present only in those previous releases of Debian and Ubuntu which carry the libvtk5.4-qt3 library. Those are – Debian squeeze (stable), Ubuntu nutty and maverick. If you upgraded your system from one of those releases, chances are that you still have fslview (and required libraries) installed although not they are not available from the APT repository anymore. Therefore fresh systems installations will not have them at all.

Workaround

While everyone is waiting for a new release of fslview compatible with Qt4 there are possible workarounds to keep research going on bleeding edge Debian-based operating systems. The first, obvious choice for a FOSS enthusiast, is to build fslview from source by first building VTK Qt3 bindings, possibly after building Qt3 itself. While it might be educationally valuable and exciting, we are afraid in the end it might be more frustrating than useful.

Therefore we would like to suggest another, much more straightforward and hopefully painless approach – lightweight virtualization, or chroot jailing, which exists in Unix-land since 1970s. With this exercise in 4 simple steps we will install a complete (minimalistic) installation of Debian stable into a separate directory – without harming the original system installation. We will provide a convenience wrapper to run fslview as if it was installed on the “main” system. So your system will stay intact while you would enhance it with additional software in a stable Debian environment. Moreover if security or critical fixes to any components of that installation become available, this chroot environment, being a complete Debian installation, could be as easily upgraded as your main system, thus guaranteeing robust performance.

Although we demonstrate this setup with fslview in mind, such approach is generally useful for various use cases. For example, we have used it in the opposite situation – on stable Debian systems we needed to run some software available only from Debian unstable or testing, and backporting of all required dependencies was either cumbersome or just impossible without sacrificing stability of the system.

Prerequisites

For this exercise you would need

3–20 minutes depending on the bandwidth to the Debian mirror and efficiency in cut/paste operations
root access to the system while performing this setup, although end-users of fslview would not need root access after everything is set up
2 additional tools – debootstrap to install Debian in a directory and a convenience utility schroot to “enable” such an environment

Procedure

Install the tools:

sudo apt-get install debootstrap schroot

Choose a location with enough space (600 MB should be enough) and install a complete Debian squeeze installation with fslview:
```
sudo debootstrap --include=fslview squeeze /srv/chroots/squeeze http://ftp.us.debian.org/debian
```
Note

You might like to adjust the URL to a Debian mirror closer to you

Create schroot configuration file /etc/schroot/chroot.d/squeeze with the following content:
```
[squeeze]
description=Debian squeeze (6.x stable)
type=directory
directory=/srv/chroots/squeeze
users=YOURLOGIN
aliases=debian,default
```
Replace YOURLOGIN with a comma separated list of users who should be allowed to access this chroot environment (see man schroot.conf for more options, e.g. how to specify whole groups with groups=..., etc.)
At this point you should already be able to invoke any command within the chroot environment, so just create a little shell script /usr/local/bin/fslview, make it executable and be all set:
```
echo -e '#!/bin/sh\nexport FSLDIR=/usr/share/fsl\nschroot -p -c squeeze /usr/bin/fslview "$@"' > /usr/local/bin/fslview
chmod a+x /usr/local/bin/fslview
```
Note

You might need to become root for the above.

Optional steps

Although at this point you can run fslview from the chroot-ed environment, we would suggest a few additional steps. For some of them (marked with chroot-root) you would need to become root in a chroot environment via following steps:

enter chroot using schroot -c squeeze -p
become root (via su command, root password should be the same as on the main system)

So here are recommended optional additions:

chroot-root: Enable NeuroDebian repository. Choose squeeze release and mirror of preference (remove sudo from provided cmdline).

chroot-root: Enable security and functionality updates:

sed -e 's,squeeze,squeeze-updates,g' /etc/apt/sources.list > /etc/apt/sources.list.d/updates.list
echo 'deb http://security.debian.org/ stable/updates main' > /etc/apt/sources.list.d/security.list
apt-get update
apt-get upgrade

Make fsl atlases accessible within the chroot environment. There are two ways and you must choose only one of them, otherwise you might damage your “main” system installation.
- chroot-root: Install atlases packages in the chroot-ed environment:
```
apt-get install fsl-atlases
```
  Although this is the best/correct way it would require additional 200MB of space, possibly duplicating what you already have installed in the main system. Also it requires enabling of NeuroDebian repository in chroot environment.
- Alternatively you can bind-mount those directories with atlases installed on the “main” system within chroot. For that edit (as root on the “main” system) /etc/schroot/default/fstab and add following entries:
```
/usr/share/fsl/data/atlases /usr/share/fsl/data/atlases none rw,bind 0 0
/usr/share/data             /usr/share/data             none rw,bind 0 0
```
  You need to be aware of the potential consequences of this second approach: Any package that installs files under /usr/share/data will modify files in the same directory outside the chroot as well. If you don’t want to risk that don’t use this method and simply install the necessary data packages inside the chroot environment too, as describe before.
  
  Note
  
  Similarly you can bind-mount any other directory you would like to make visible in chroot. Just be careful to not “overlap” with system directories in chroot which already carry something.

Also you might like to read man schroot on how to enable persistent sessions so that chroot initiation could be done ones during boot instead of per each fslview invocation

If you have any comments (typos, improvements, etc) – feel welcome to leave a comment below, or just email us@NeuroDebian .

NeuroDebian@HBM2011.ca

team@neuro.debian.net (NeuroDebian team) — Sun, 10 Jul 2011 20:00:00 -0000

NeuroDebian@HBM2011.ca

On June 26-30 the annual meeting of the Organization for Human Brain Mapping (HBM2011) took place in Quebec City, Canada. Encouraged by our positive experience at last year’s SfN in San Diego and enthusiasm of our scientific adviser, James V. Haxby, we hosted another NeuroDebian booth. The setup was pretty much the same as last year: Some chairs and tables, lots of people, our tri-fold flyers, a Debian mirror and some virtual machine images to show Debian in action. This time we also had an LCD display attracting visitors with the package swarm, some demos, and our recent paper. We had many curious people have their first exposure to Debian, long-time users expressing their gratitude to Debian, and our upstream developers getting together to discuss various topics. Having registered the booth as “NeuroDebian”, we had the additional pleasure of explaining visitors the concept of a project inside Debian, in contrast to a derived distribution. But that is nothing new really, so let’s talk about the differences from last year’s booth.

First of all, we had more people at the booth. Dominique Belhachemi volunteered to help us out – and that was very much appreciated. Although HBM has only about a tenth of the attendees that SfN has, we had significantly more traffic. While last year people were primarily interested in knowing about the project, this time many of them wanted to give it a try immediately. People came with their laptops, got the VM images and started playing with Debian. After a day or so, some came back and asked for recommendations on particular software – after having been exposed to the wealth of the Debian archive.

What also had increased was the number of developers, or rather research labs developing neuroimaging software that came to the booth to discuss how to get their software into Debian and how to arrange ongoing maintenance of these future Debian packages. As we have our plates already quite full, we have been spending some time on mentoring interested developers to learn the art of Debian packaging and making them familiar with Debian’s procedures and standards (e.g. working on #609820 with Yannick Schwartz, upstream, at the booth).

Two promising new developments need to be mentioned. First, we were approached by companies that develop hardware for brain-imaging and psychophysics research. They were curious to learn about Debian as an integrated platform that offers free software solutions that an increasing amount of their customers demands (e.g. PsychoPy). Apparently, the movement towards open research software has finally made it into the business plans of companies, as they seem to start perceiving compatibility with free software systems as a competitive advantage. We explained how software gets into Debian, and how its release cycle is managed. To foster their motivation we also pointed them to the existing open-source software that is already available or even present in Debian. Let’s see whether we see more “Debian-certified” research products in the future.

Lastly, we started talking with folks from the INCF to explore possibilities of collaborating on INCF projects using Debian as the integration and development platform. The INCF is an OECD-funded organization that develops collaborative neuroinformatics infrastructure and promotes the sharing of data and computing resources to the international research community. At least one INCF project is already relying on the efforts of the NeuroDebian project. We are going to continue this discussion during a workshop in September. A report will follow...

Debian people at the booth (f.l.t.r): Michael Hanke, Yaroslav Halchenko, Stephan Gerhard, Dominique Belhachemi. Not shown: Swaroop Guntupalli.

Acknowledgments

This booth has been made possible by the generous support of Prof. James V. Haxby (Dartmouth College, New Hampshire, USA).

Neuroscience runs on GNU/Linux

team@neuro.debian.net (NeuroDebian team) — Mon, 27 Jun 2011 23:00:00 -0000

Neuroscience runs on GNU/Linux

While everybody else is waiting for Linux to become the standard on the desktop, in the field of neuroscience research we have made this step already – so silently that we haven’t even realized it ourselves.

In the NeuroDebian project we were trying to figure out what software would need to be integrated into Debian to have a maximum impact on enhancing its utility as a research platform. So we ran a survey, asking people to tell us what software they use in their research and what platform they run it on. We also asked people to describe their personal preferences regarding a computing environment and how that might differ from the computing platforms they are institutionally provided with.

The main result of this survey was very simple and pretty surprising (even to us): Despite common believe, GNU/Linux is the current standard computing platform in neuroscience – and that is both on the big compute clusters and on the laptop and desktop. It turns out that researchers on GNU/Linux are not only the majority, they are also much happier with what they get, and spend less time on non-research maintenance tasks than the guys who are still stuck with Windows.

It was also nice to see that Debian-based systems are preferred by neuroscientists among GNU/Linux distributions for their personal computing environments and that even proportions of Red Hat and Debian-based systems together represent the vast majority of GNU/Linux deployments in institutional computing infrastructure.

Well done Debian!

The results have just been accepted for publication in the journal Frontiers in Neuroinformatics. A link to the manuscript, the original survey form, collected data, as well as supplementary analysis results are available at: http://neuro.debian.net/survey/2011/

Package swarm

team@neuro.debian.net (NeuroDebian team) — Mon, 20 Jun 2011 20:00:00 -0000

Package swarm

We are working on integrating neuroscience research software into the Debian operating system for almost six years now. Occasionally looking at the repository web server logs, we knew that a growing number of people were using the packages from http://neuro.debian.net for their Debian and Ubuntu machines. However, we never really looked deeper.

For our upcoming booth at HBM2011 we need something like a presentation or slides that could run in a loop on a big screen to attract people. It would need to be visually appealing, but also informative. As slides are rather boring, we looked into creating an animation – a visualization of our repository download statistics.

There are many boring ways to visualize this type of data, but there is also at least one that is cool enough to yield an appealing, almost mystical, animation: code_swarm. This great software can visualize somewhat “social” processes based on commit statistics of source code repositories. It extracts information on who is working on which files of a project at any given point in time, and visualizes similarities in these patterns. People that work on the same files in temporal proximity cluster together, and files that are modified by the same people cluster together. From all this information code_swarm generates beautiful animations.

Download statistics are not that different from commit statistics. Instead of “Who is modifying what?” it is simply: “Who is downloading what?” So we gathered all Apache logs of binary package downloads from our repository over the last two years. We converted the data so that the corresponding source package, became the “author” of a commit/download, and encoded the associated IPv4 address (the last 8 bit truncated) together with the target distribution as the “modified file”. And code_swarm did the rest (try watching in 480p or above):

Few notes on the video: The animation shows some spikes in the download activity. Most of the big ones are related to new releases of FSL. FSL is undoubtedly our most popular package. First and foremost, because it is one of the standard tools for brain-imaging research, but also, because upstream completely relies on NeuroDebian for deploying FSL on the Debian platform.

It looks like the majority of our downloads comes from countless Ubuntu machines – maybe laptops and desktops with dynamic IPs. However, there are also fairly/large deployments of Debian 6.0 (white) and Ubuntu 10.04 (light blue) with static addresses – probably labs at research institutions. Watch for the large “stars” towards the end.

You might have noticed the yellow dots. These are downloads from the data suite of the NeuroDebian which we use to improve space-efficiency of our repository. This suite has packages (sometimes the size of 1GB in compressed form) that can be installed on any Debian or Ubuntu release and contain only data of various types and formats. Of course many applications depend on these packages, hence you can see some dots constantly changing color between yellow and one of the release colors.

PS: OpenShot became an amazing video editor!

Debian booth at SfN2010 in San Diego

team@neuro.debian.net (NeuroDebian team) — Wed, 24 Nov 2010 14:00:00 -0000

Debian booth at SfN2010 in San Diego

During November 13-17, 2010 the NeuroDebian team ran its first Debian booth at the annual meeting of the Society for Neuroscience (SfN2010) in San Diego, USA. We presented the upcoming release Debian 6.0 Squeeze and demonstrated its utility as a robust and versatile research environment for neuroscience. Booth visitors had the opportunity to meet with developers of neuroscience research software, and to get information on available software and recommendations for deployment strategies in research laboratories.

The annual meeting of the Society for Neuroscience is one of the largest neuroscience conferences in the world, with over 30,000 attendees. Researchers, clinicians, and leading experts discuss the latest findings about the brain, nervous system, and related disorders.

Booth setup

Don Armstrong kindly provided us with Debian banners to decorate the booth and some Debian T-shirts to give away. Moreover, we were equipped with laptops running Debian squeeze and sid, as well as two additional laptops each running a Debian squeeze virtual machine on top of Mac OS X and Windows, respectively (CDs with the VM image were also available for visitors to take home). To demonstrate Debian’s versatility, we had a complete Debian archive mirror that was used to show the full selection of available software and the simplicity of installation and upgrade procedures. The mirror was provided from an external harddrive by a commodity router box running the Debian-based DebWrt distribution. All machines were connected to our own local wired network to avoid problems with conference center’s free wireless network (poor at best). Finally, we had several hundred tri-fold flyers with general Debian facts on one side, and NeuroDebian facts on the other (sources are available).

Final booth setup with staff (left to right): Michael Hanke, Yaroslav Halchenko and Swaroop Guntupali.

Booth visitors

The booth was well attended on all days of the conference. Many people were somewhat surprised, but also pleased to see Debian represented. The visitors comprised the whole range from long-term Debian users to people who were not aware of an operating system other than Windows and Mac OS.

A number of visitors were involved in free software development – at various levels. We talked to a Debian ftpmaster, a Gentoo developer, various developers of neuroscience-related software that is already integrated in Debian and many more whose work still needs to be packaged. We were visited by representatives of companies looking for support to get their open-source products into Debian. The vast majority, however, were scientists looking for a better research platform for their labs. That included the struggling Phd-student, as well as lab heads sharing their experience managing a computing infrastructure for neuroscience research.

The Debian booth also served as a platform for upstream developers to meet with Debian users of their software.

Debian-based systems are the preferred Linux environment

The overwhelming majority of visitors running some Linux flavor used a Debian-based operating system – including Debian itself, Ubuntu and sidux/apttosid. Especially people using Python for research purposes seem to prefer the comprehensive support of Python modules in Debian, whereas e.g. R users are more uniformly distributed across GNU/Linux distributions. This assessment is, of course, biased by the fact that Debian was the only distribution that was present at this conference.

In general, we had the impression that Linux users employ a larger variety of tools in their research activities, whereas users of proprietary operating systems tended to limit themselves to a more restricted set, or use an intermediate computing platform, such as Matlab.

During the conference’s poster session we explained the Debian system and community processes to many interested visitors (download the poster).

Take-home messages

While there was a large variety of topics that were brought up by visitors there were some common patterns.

Visibility

For a Debian developer it may be surprising, but many people still do not know that Debian exists – even despite the fact that Debian is often a perfect match for their particular requirements. People who got introduced to Debian at the booth often couldn’t believe what they were hearing or seeing: so much software, runs on any hardware, all for free.

We believe it would be very beneficial for Debian to reach out beyond the IT sector and present itself in all fields of applications that it already supports today.

Debian and Ubuntu

Apparently it is still a largely unknown fact that Ubuntu is based on Debian.

Live-CD

There was a significant demand for (customized) Live-CDs. On one hand, people were asking for a way to quickly try out Debian (and we believe that this doesn’t necessarily have to be a live-cd). On the other hand, for example, teachers were asking for means to temporarily deploy Debian on, e.g. university computer pool machines and use Debian-packaged software for teaching courses (e.g. on brain-imaging data analysis).

Electrophysiology tools

Of all subfields of neuroscience, electrophysiology researchers expressed the greatest demand for better tools in Debian – or basically at least some specialized tools at all. Moreover, many research projects relying on FOSS solutions in electrophysiology already use Debian-based systems to accomplish the mission; they just rely on manual (from sources) deployment of the necessary tools. We started a new Debian Science Blend task to collect information about existing relevant software and to eventually package it.

Realtime capabilities

Apparently, numerous research groups utilize Debian-based equipment to perform various flavors of real-time data acquisition and processing. They expressed their demand for real-time capabilities of (some) Debian kernel images.

Cloud-computing

Cloud-computing seems to be an increasingly interesting topic for neuroscience data analysis. We got the impression that there is a tendency to look for alternatives to Matlab to be able to run analyses in the cloud cheaper (or at all). We pointed people to ongoing efforts in Debian to enable Debian-based cloud computing (see e.g. the Debian wiki).

Many Thanks

Throughout the conference many people stopped by to express their gratitude to Debian for developing their operating system of choice. We want to affirm this and relay it to the larger Debian community. Thanks for Debian.

Acknowledgements

This booth has been made possible by the generous support of Prof. James V. Haxby (Dartmouth College, New Hampshire, USA) and other donations to the Debian project.