I wanted to use VNC to allow multiple desktops on a server.
In Ubuntu 16.04 (or 14.04, 17.10 or Linux Mint Cinnamon) I found that neither the GNOME desktop nor the XFCE desktop would work with VNC - I tried vnc4server, realvnc, and tigervnc. They all crashed in slightly different ways, but each was unuseable.
The solution, which so far I've tested in Mint and 16.04 with tigervnc, is just to make sure you uninstall the default vino desktop sharing tool before installing tigervnc. Simple!
Saturday, 18 November 2017
Thursday, 18 August 2016
Weird matlab behaviour for empty matrices.
I've noticed that (at least in Matlab 2015b) a weird behaviour with Matlab matrices.
Here are two ways of making different empty matrices:
>> a=[]
>> size(a)
ans=
0 0
>> isempty(a)
ans=
1
>> b=find(1==[0 0])
>> size(b)
ans=
1 0
>> isempty(b)
ans=
1
This is a pernicious problem. Is there any reason for it to be like this?
Here are two ways of making different empty matrices:
>> a=[]
>> size(a)
ans=
0 0
>> isempty(a)
ans=
1
>> b=find(1==[0 0])
>> size(b)
ans=
1 0
>> isempty(b)
ans=
1
They both look empty, but have different sizes. But critically for some code, they behave very differently in a for loop
>> for ind=a
>> ...this never gets executed!
>> end;
>> for ind=b
>> ...this gets executed once, with ind=[];
>> end;
Monday, 21 April 2014
Illustrator revalation
I often use Adobe Illustrator to draw illustrations or finesse graphics. While I'd prefer a cheaper package, there doesn't currently seem to be anything that comes close to the sophistication of this tool. Like Photoshop (or Flash) it isn't the most intuitive thing, though. You really need expertise. But first, you need to know what the feature is that you need. As soon as you know this, Google is your friend.
So, today, I thought "Wouldn't it be cool if there were a way to align multiple objects not just to each other, but to the fixed position of a single object?"
It turns out this is dead simple - you shift-click to select all the objects you wish to align, and then do an extra click on one of these selected items. This becomes the reference to which everything else is aligned.
I was so excited I had to share.
So, today, I thought "Wouldn't it be cool if there were a way to align multiple objects not just to each other, but to the fixed position of a single object?"
It turns out this is dead simple - you shift-click to select all the objects you wish to align, and then do an extra click on one of these selected items. This becomes the reference to which everything else is aligned.
I was so excited I had to share.
Thursday, 3 April 2014
Disabling Chrome Viewer for PDFs...
...because Preview on a Mac does a much nicer job of rendering....
http://www.cnet.com/how-to/how-to-disable-chromes-pdf-viewer/
http://www.cnet.com/how-to/how-to-disable-chromes-pdf-viewer/
Sunday, 27 October 2013
meld
Both for maintaining github repositories, and assorted code merging, I've been enjoying using the tool "meld". For adhoc merging, if you have two different versions of a file, do
meld [filename1] [filename2]
You'll then see the two files in parallel, with MELD highlighting new, changed or deleted code in each file. You can then click the arrows to transfer the change in one direction or the other.
You may also merge more than two versions.
If you use git, you may set this to use meld by default:
git config --global merge.tool meld
Then type
git mergetool
will resolve all of the conflicts that have arisen with an update.
meld [filename1] [filename2]
You'll then see the two files in parallel, with MELD highlighting new, changed or deleted code in each file. You can then click the arrows to transfer the change in one direction or the other.
You may also merge more than two versions.
If you use git, you may set this to use meld by default:
git config --global merge.tool meld
Then type
git mergetool
will resolve all of the conflicts that have arisen with an update.
Saturday, 5 October 2013
Benchmarking ZFS against LVM
ZFS is a filing system that offers deduplication (i.e., two copies of the same file in different parts of the file system are only stored once) and compression with two algorithms (lzjb or gzip). A trial dataset of typical neuroimaging data from our system (59GB ) gave a 1.5 deduplication ratio and compression of 1.25 with lzjb and 1.32 with gzip. These are tests of ZFS backed by EBS on an Amazon EC2 cc2.8xlarge server.
Following a clear of the memory cache with:
sync; echo 3 > /proc/sys/vm/drop_caches
I then benchmarked a directory listing (~1000 nifti files) and read/write (10 random EPI files per subject). Access times (ms) with statistics over 10 subjects of 4 sessions:
dir zfs-nocompress 58.8 +/- 42.0 zfs-lzjb 38.5 +/- 16.2 zfs-gzip 53.9 +/- 47.6 lvm 146.8 +/- 480.6
read zfs-nocompress 576.3 +/- 277.0 zfs-lzjb 629.7 +/- 301.7 zfs-gzip 523.5 +/- 82.1 lvm 499.8 +/- 377.1
write zfs-nocompress 207.1 +/- 13.1 zfs-lzjb 207.0 +/- 12.6 zfs-gzip 205.5 +/- 6.5 lvm 188.3 +/- 1.3
On a second run immediately succeeding the one above (to maximize caching)
dir zfs-nocompress 6.0 +/- 0.3 zfs-lzjb 5.8 +/- 0.4 zfs-gzip 5.9 +/- 0.3 lvm 4.8 +/- 0.2
read zfs-nocompress 549.8 +/- 150.6 zfs-lzjb 601.4 +/- 292.7 zfs-gzip 544.2 +/- 210.8 lvm 461.2 +/- 200.0
write zfs-nocompress 207.8 +/- 17.3 zfs-lzjb 206.9 +/- 16.4 zfs-gzip 203.7 +/- 1.9 lvm 189.8 +/- 3.1
Directory listing has got faster, and reads a bit faster.
ZFS seems slower for reading (1-2.5 times, depending on conditions), partly due to decompression latencies. It also uses more CPU, but from watching the system monitor, this was barely noticeable here as the disk access to EBS is still the bottleneck.
ZFS has other benefits - it supports caching (on SSD drives for cc2.8xlarge ephemerals); and snapshots for backup, which don't use memory except to store changes in files.
In summary, for our data deduplication and compression look valuable, but it does come with a performance cost. But, this can be made up for by more parallelisation of jobs. gzip looks the better option for our data.
Update: After copying in ~3TB of data, in lots of small files, ZFS appears to be slowing down for both directory listing and reading. The files had probably fallen out of the cache (unless ZFS had kept them in because of their frequency of access).
dir zfs-nocompress 35.8 +/- 33.6 zfs-lzjb 86.2 +/- 51.6 zfs-gzip 127.7 +/- 179.2 lvm 5.1 +/- 0.2
read zfs-nocompress 792.1 +/- 415.7 zfs-lzjb 822.9 +/- 372.7 zfs-gzip 887.2 +/- 475.5 lvm 390.0 +/- 36.3
write zfs-nocompress 209.5 +/- 2.6 zfs-lzjb 208.5 +/- 2.1 zfs-gzip 208.1 +/- 3.7 lvm 192.5 +/- 1.4
And a repeat, where the files will have been in the cache:dir zfs-nocompress 19.2 +/- 8.2 zfs-lzjb 23.3 +/- 14.2 zfs-gzip 22.0 +/- 19.5 lvm 5.0 +/- 0.2
read zfs-nocompress 781.9 +/- 429.4 zfs-lzjb 698.2 +/- 179.7 zfs-gzip 788.0 +/- 320.0 lvm 409.9 +/- 135.7
write zfs-nocompress 207.8 +/- 3.3 zfs-lzjb 206.4 +/- 2.2 zfs-gzip 207.5 +/- 5.0 lvm 193.3 +/- 2.5
ZFS is doing even worse. This seems worrying... does anyone have a solution?
Update: Under reasonable load (here 5 simultaneous copies from lvm to zfs) things really seem to fall apart with ZFS:
dir zfs-nocompress 4438.2 +/- 20030.3 zfs-lzjb 453.5 +/- 554.9 zfs-gzip 732.9 +/- 530.3 lvm 14.3 +/- 10.5
read zfs-nocompress 8622.1 +/- 5483.3 zfs-lzjb 5494.3 +/- 4014.2 zfs-gzip 1485.2 +/- 824.7 lvm 445.9 +/- 198.3
write zfs-nocompress 4400.4 +/- 4677.5 zfs-lzjb 1946.9 +/- 3082.8 zfs-gzip 2445.3 +/- 2301.6 lvm 201.5 +/- 13.5
I've checked the L2ARC cache and it is nowhere near full - 8 GB out of 32 GB max. What's the bottleneck, and why does it degrade so dramatically?
Update: Part of the bottleneck is de-duplication, as described in this post:
http://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSDedupMemoryProblem?showcomments#comments
Switching this off speeds things up, but not enough under reasonable load - ZFS is still unacceptably slow.
Conclusion: Abandoned as a solution for now, until I can identify the cause of the slowing.
Following a clear of the memory cache with:
sync; echo 3 > /proc/sys/vm/drop_caches
I then benchmarked a directory listing (~1000 nifti files) and read/write (10 random EPI files per subject). Access times (ms) with statistics over 10 subjects of 4 sessions:
dir zfs-nocompress 58.8 +/- 42.0 zfs-lzjb 38.5 +/- 16.2 zfs-gzip 53.9 +/- 47.6 lvm 146.8 +/- 480.6
read zfs-nocompress 576.3 +/- 277.0 zfs-lzjb 629.7 +/- 301.7 zfs-gzip 523.5 +/- 82.1 lvm 499.8 +/- 377.1
write zfs-nocompress 207.1 +/- 13.1 zfs-lzjb 207.0 +/- 12.6 zfs-gzip 205.5 +/- 6.5 lvm 188.3 +/- 1.3
On a second run immediately succeeding the one above (to maximize caching)
dir zfs-nocompress 6.0 +/- 0.3 zfs-lzjb 5.8 +/- 0.4 zfs-gzip 5.9 +/- 0.3 lvm 4.8 +/- 0.2
read zfs-nocompress 549.8 +/- 150.6 zfs-lzjb 601.4 +/- 292.7 zfs-gzip 544.2 +/- 210.8 lvm 461.2 +/- 200.0
write zfs-nocompress 207.8 +/- 17.3 zfs-lzjb 206.9 +/- 16.4 zfs-gzip 203.7 +/- 1.9 lvm 189.8 +/- 3.1
Directory listing has got faster, and reads a bit faster.
ZFS seems slower for reading (1-2.5 times, depending on conditions), partly due to decompression latencies. It also uses more CPU, but from watching the system monitor, this was barely noticeable here as the disk access to EBS is still the bottleneck.
ZFS has other benefits - it supports caching (on SSD drives for cc2.8xlarge ephemerals); and snapshots for backup, which don't use memory except to store changes in files.
In summary, for our data deduplication and compression look valuable, but it does come with a performance cost. But, this can be made up for by more parallelisation of jobs. gzip looks the better option for our data.
Update: After copying in ~3TB of data, in lots of small files, ZFS appears to be slowing down for both directory listing and reading. The files had probably fallen out of the cache (unless ZFS had kept them in because of their frequency of access).
dir zfs-nocompress 35.8 +/- 33.6 zfs-lzjb 86.2 +/- 51.6 zfs-gzip 127.7 +/- 179.2 lvm 5.1 +/- 0.2
read zfs-nocompress 792.1 +/- 415.7 zfs-lzjb 822.9 +/- 372.7 zfs-gzip 887.2 +/- 475.5 lvm 390.0 +/- 36.3
write zfs-nocompress 209.5 +/- 2.6 zfs-lzjb 208.5 +/- 2.1 zfs-gzip 208.1 +/- 3.7 lvm 192.5 +/- 1.4
And a repeat, where the files will have been in the cache:dir zfs-nocompress 19.2 +/- 8.2 zfs-lzjb 23.3 +/- 14.2 zfs-gzip 22.0 +/- 19.5 lvm 5.0 +/- 0.2
read zfs-nocompress 781.9 +/- 429.4 zfs-lzjb 698.2 +/- 179.7 zfs-gzip 788.0 +/- 320.0 lvm 409.9 +/- 135.7
write zfs-nocompress 207.8 +/- 3.3 zfs-lzjb 206.4 +/- 2.2 zfs-gzip 207.5 +/- 5.0 lvm 193.3 +/- 2.5
ZFS is doing even worse. This seems worrying... does anyone have a solution?
Update: Under reasonable load (here 5 simultaneous copies from lvm to zfs) things really seem to fall apart with ZFS:
dir zfs-nocompress 4438.2 +/- 20030.3 zfs-lzjb 453.5 +/- 554.9 zfs-gzip 732.9 +/- 530.3 lvm 14.3 +/- 10.5
read zfs-nocompress 8622.1 +/- 5483.3 zfs-lzjb 5494.3 +/- 4014.2 zfs-gzip 1485.2 +/- 824.7 lvm 445.9 +/- 198.3
write zfs-nocompress 4400.4 +/- 4677.5 zfs-lzjb 1946.9 +/- 3082.8 zfs-gzip 2445.3 +/- 2301.6 lvm 201.5 +/- 13.5
I've checked the L2ARC cache and it is nowhere near full - 8 GB out of 32 GB max. What's the bottleneck, and why does it degrade so dramatically?
Update: Part of the bottleneck is de-duplication, as described in this post:
http://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSDedupMemoryProblem?showcomments#comments
Switching this off speeds things up, but not enough under reasonable load - ZFS is still unacceptably slow.
Conclusion: Abandoned as a solution for now, until I can identify the cause of the slowing.
Monday, 5 August 2013
For UWO employees... accessing journal or book subscriptions from outside the office
If you've gone through pubmed or Scholar and arrived at a paper you'd like to obtain, but you've been rejected because the site doesn't recognize your subscription, then just put "proxy2.lib.uwo.ca" onto the end of the site name, between the URL and its parameters. For example...
http://www.sciencedirect.com/science/article/pii/S1053811910010062
becomes...
http://www.sciencedirect.com.proxy2.lib.uwo.ca/science/article/pii/S1053811910010062
You'll then need your Western ID and password, of course.
Magic, huh?
http://www.sciencedirect.com/science/article/pii/S1053811910010062
becomes...
http://www.sciencedirect.com.proxy2.lib.uwo.ca/science/article/pii/S1053811910010062
You'll then need your Western ID and password, of course.
Magic, huh?
Tuesday, 30 July 2013
Setting up your .bashrc for Mturk on cercimage
Add these two lines on the end:
export PATH=/home/rcusack/software/aws-mturk-clt-1.3.0/bin:$PATH
export MTURK_CMD_HOME=/home/rcusack/software/aws-mturk-clt-1.3.0
export PATH=/home/rcusack/software/aws-mturk-clt-1.3.0/bin:$PATH
export MTURK_CMD_HOME=/home/rcusack/software/aws-mturk-clt-1.3.0
Sunday, 28 July 2013
Thursday, 25 July 2013
Poisson disc in Matlab
Poisson disc sampling is a nice way of getting random yet even coverage.
Following the helpful page from devmag here are 2d and 3d implementations of Poisson disc (sphere?) sampling in Matlab. Also an implementation using SPM's grey mask to parcellate grey matter to an arbitrary degree.
Following the helpful page from devmag here are 2d and 3d implementations of Poisson disc (sphere?) sampling in Matlab. Also an implementation using SPM's grey mask to parcellate grey matter to an arbitrary degree.
Wednesday, 20 February 2013
Get rid of Unity under RealVNC
Go to the directory .vnc in your home directory.
Backup the file xstartup, make a new file, and replace it with:
#!/bin/sh
# Uncomment the following two lines for normal desktop:
# unset SESSION_MANAGER
# exec /etc/X11/xinit/xinitrc
#[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
xsetroot -solid grey
vncconfig -iconic &
#xterm -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &
touch /home/rhodri/.vnc/haverunxstarup
gnome-session --session=gnome-classic &
Backup the file xstartup, make a new file, and replace it with:
#!/bin/sh
# Uncomment the following two lines for normal desktop:
# unset SESSION_MANAGER
# exec /etc/X11/xinit/xinitrc
#[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
xsetroot -solid grey
vncconfig -iconic &
#xterm -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &
touch /home/rhodri/.vnc/haverunxstarup
gnome-session --session=gnome-classic &
On restarting your vncserver, you should get an old style desktop.
Thursday, 3 January 2013
Ubuntu reanimates netbook
I had an essentially dead Acer Aspire One D257 netbook. With Windows 7 Starter, which it arrived with out of the box (along with lots of bloatware) it had become unbearably slow. There were two options: dustbin, or reanimation with Linux. After a failed diversion with Joli OS (which is very pretty but the Wifi resolutely refused to work), I installed Ubuntu 12.10. This works very immediately, and has made the machine a pleasure to use. I also found the preview version of Spotify for Linux, which seems to work well too.
Tuesday, 23 October 2012
Matlab/FSL + Ubuntu 12.04 library problem
Since upgrading to Ubuntu Precise 12.04, we've been getting an error on starting matlab of
/usr/local/MATLAB/R2011b/bin/util/oscheck.sh: /lib64/libc.so.6: not found
This same problem also seemed to break bet from FSL.
Now in your matlab path, for us
/usr/local/MATLAB/R2011b/sys/os/glnxa64
you'll find
libstdc++.so.6
symbolically linked to
libstdc++.so.6.0.1.13
Ubuntu ships with a newer version of this library in
/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16
So, I symbolically linked the Matlab version through to the one shipped with Ubuntu
$ cd /usr/local/MATLAB/R2011b/sys/os/glnxa64
$ mv libstdc++.so.6.0.1.13 libstdc++.so.6.0.1.13.old
$ ln -s /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16 libstdc++.so.6.0.1.13
This seems to fix it.
Friday, 5 October 2012
Tuesday, 21 August 2012
Installed iBeat
I've installed the iBeat infant brain image processing software on ssc-cercimage.
To use it, add to your .bashrc:
export IBEAT_HOME=/home/rcusack/software/ibeat
source /home/rcusack/software/ibeat/ibeatEnv.bash
To use it, add to your .bashrc:
export IBEAT_HOME=/home/rcusack/software/ibeat
source /home/rcusack/software/ibeat/ibeatEnv.bash
Sunday, 5 August 2012
Using Ubuntu 12.04 by VNC
I find the Unity desktop that ships with Ubuntu is clumsy to use over VNC. When changing workspaces, for example, it shows a full screen overview of what is on each desktop, which takes a long time to load over a slower connection. I'm sure it is very beautiful when sitting in front of the machine, but it is grating when used over a remote connection.
Also, there seem to be some exotic keystrokes (e.g., ctrl+alt+arrow to switch workspaces), or lots of middle and right button work on the taskbar, which often don't translate properly down a VNC.
To switch desktops, first install gnome with
apt-get install gnome-session-fallback
Then, to configure VNC to launch into this, change the file .vnc/xstartup so that it has the line:
exec gnome-session --session=gnome-classic
Also, there seem to be some exotic keystrokes (e.g., ctrl+alt+arrow to switch workspaces), or lots of middle and right button work on the taskbar, which often don't translate properly down a VNC.
To switch desktops, first install gnome with
apt-get install gnome-session-fallback
Then, to configure VNC to launch into this, change the file .vnc/xstartup so that it has the line:
exec gnome-session --session=gnome-classic
Tuesday, 10 July 2012
ec2 & fstab
If you're setting up an ec2 instance in the Amazon cloud and are setting devices (EBS volumes) to mount on reboot by adding them to /etc/fstab, make sure you put "nobootwait" in the options, like this:
/dev/xvdh /export/home ext3 auto,rw,nobootwait 0 0
If you don't, then if the drive isn't present, the boot process will hang, either with a blank console output, or with:
The disk drive for xxx is not ready yet or not present
Continue to wait; or Press S to skip mounting or M for manual recovery
and as the keyboard is miles away, you can't press that key.
/dev/xvdh /export/home ext3 auto,rw,nobootwait 0 0
If you don't, then if the drive isn't present, the boot process will hang, either with a blank console output, or with:
The disk drive for xxx is not ready yet or not present
Continue to wait; or Press S to skip mounting or M for manual recovery
and as the keyboard is miles away, you can't press that key.
Tuesday, 27 March 2012
Monday, 26 March 2012
Configure remote access to git without erasing your existing keys
This is a solution if you'd like to connect to github using ssh (typical for read-write access to a repository) but you don't want to delete an existing ssh key (e.g., id_rsa) as requested in the git instructions. On ubuntu, I did it like this:
Copy the public key file ~/.ssh/github_rsa.pub to your github.com ssh settings.
Edit the file .ssh/config, and add the following entry (or create a new file if it doesn't currently exist)
Host github
Hostname github.com
User git
IdentityFile ~/.ssh/github_rsa
Then, I had to do
Then, if you do
you should see
Hi rhodricusack! You've successfully authenticated, but GitHub does not provide shell access.
Then you can do
git clone github:/rhodricusack/automaticanalysis
$ cd .ssh
$ ssh-keygen -t rsa -C "myemail@place.com"
Enter file in which to save the key (/home/username/.ssh/id_rsa): github_rsa
Enter a passphrase: your secret phrase
Enter a passphrase: your secret phrase
Copy the public key file ~/.ssh/github_rsa.pub to your github.com ssh settings.
Edit the file .ssh/config, and add the following entry (or create a new file if it doesn't currently exist)
Host github
Hostname github.com
User git
IdentityFile ~/.ssh/github_rsa
Then, I had to do
$ ssh-add ~/.ssh/github_rsa
to get rid of an "Agent can't sign error"Then, if you do
$ ssh -T github
you should see
Hi rhodricusack! You've successfully authenticated, but GitHub does not provide shell access.
Then you can do
git clone github:/rhodricusack/automaticanalysis
Sunday, 18 March 2012
Using Univa grid engine for FSL
GridEngine is installed on cercimage at the BMI. Elsewhere, you'll need to install the Univa GridEngine. Then, add the following line to the end of the .bashrc file in your home directory
. /gridware/univa/default/common/settings.sh
A number of FSL functions (including POSSUM & MELODIC) will then execute in parallel. You may type qstat or "watch qstat" to see the jobs being queued up.
. /gridware/univa/default/common/settings.sh
A number of FSL functions (including POSSUM & MELODIC) will then execute in parallel. You may type qstat or "watch qstat" to see the jobs being queued up.
Subscribe to:
Posts (Atom)
