Here is a simple solution, a combination of https://unix.stackexchange.com/a/341277/25975 and https://unix.stackexchange.com/a/28520/25975

Add the following functions into your ~/.bashrc It will push the current title and icon into a stack and pop it afterwards.

Restart bash / log out and back in, and it should work.

For security reasons, it is not possible to query the current title of the terminal. However, with the following command, you can push the current one on to a stack

The title can then be set to anything, by ssh for example. You can then pop that back from the stack using

The main step is to go into the settings in Docker Desktop -> Resources and make sure that your distribution is enabled for docker.

1. Make sure that you have no docker packages installed on your WSL distribution. Docker Desktop will deploy its own binaries, and any pre-existing binaries will confuse it. This issue exhibited itself for me with errors related to missing files around credentials.
2. Remove any DOCKER_HOST environment variables. Docker Desktop will sort it out. Docker kept hanging for me until I fixed this.
3. If you want to use docker as non-root user, add yourself to the docker group.

Errors / Issues I ran into:

docker.credentials.errors.InitializationError: docker-credential-desktop.exe not installed or not available in PATH - Fixed by 1 above.

docker-compose from WSL2 errors out - Again, fixed by 1

Unable to run docker as non-root user - fixed by 3.

Docker hangs when run as non-root user - fixed by 2.

After much head scratching and metaphorically banging my head against a wall, here are the things I learnt.

Empty Trash

Local Trash

Make sure that have emptied the trash on the VMs .Ubuntu has this issue and so might other distributions

Network Trash

If you have SAMBA enabled on your VMs make sure that the Recycle Bin is not enabled. I have openmediavault running on a VM and I had to go through and disable the Recycling Bin. Make sure that the Recycle bin is emptied. They are hidden folders in the root of your shares.

Correct Driver & Settings

• When setting up the hard drive for your VM, make sure you use virtio-scsi (or just scsi on the web interface).
  • If you disk is already set up using IDE or VirtIO,
    • Delete it. Don't worry, it's only deleting the link. The disk itself will show up in the interface afterwards
    • Double click on the unattached disk and select SCSI and Discard
    • You might have to fix the references to the drive in the OS
• On the Device Screen, make sure discard is selected.

TRIM

Configure the OS to send TRIM commands to the drive

Linux

On Mount

You can pass the parameter discard to any mountpoint and the correct TRIM commands will be sent to the disk. HOWEVER, this is apparently a big performance hit.

To do the actual trim, run

OR to run fstrim on all supported drives

Digital Ocean has a detailed post about setting TRIM and setting up a schedule etc.

Windows

My condolences! Also, I don’t run Windows on any of my VM’s so I have no experience with it.

I set up a pool using 4 2TB SATA disks.

The total size displayed here is the total size of the 4 disks. The maths works as 4*2TB = 8TB = ~7.25TiB

RAIDZ2 is like RAID6 and it uses two disks for parity. Thus, I would expect to have ~4TB or 3.63TiB of available space. I haven’t been able to find this number displayed anywhere.

However, you can find the amount of disk space still available using the following command.

The value of 2.19T is the amount of unallocated space available in the pool. To verify this, you can run

If we add the two numbers here, 1.21T + 2.19T = 3.4T.

5% of disk space is reserved, so 3.63 * 0.95 = 3.4T

et voila

Firstly, create a new file called /etc/group.login.allow (it can be called anything - you just need to update the line below to reflect the name)

In this file, pop in all the groups that should be able to login

Edit /etc/pam.d/common-auth (in ubuntu), it might be called /etc/pam.d/system-auth or something else very similar. At the top of the file (or at least above other entries, add the following line:

For the record, found this little tidbit over at the centos forums\

I used this to run sl (yes sl, not ls - try it - I love it) repeatedly.

for example

Bear in mind that this loop is infinite and there is no way to cancel out of it except to kill of the terminal.

The first thing that you’ll want to do is to add in bricks. Bricks are similar to physical volumes a la LVM. The thing to bear in mind is that depending on what type of cluster you have (replicated / striped), you will need to add a certain number of blocks at a time.

Once you have a initialised the nodes, to add in a set of bricks, you need the following command which adds two more bricks to a cluster which keeps two replicas.

Once you have done this, you will need to rebalance the cluster, which involves redistributing the files across all the bricks. There are two steps to this process, the “fixing” of the layout changes and the rebalancing of the data itself. You can perform both tasks together.

As a starting point, to view the status of a rebalance, you can use:

You can also stop / pause a rebalance with

To “fix” the layout changes, you need to run:

Rebalancing the volume to migrate the data is easy and can be done using a similar command:

To complete both in one command, you just need:

Easy right?

With this mechanism, you have the ability to have storage that can be expanded on the fly by using additional hardware. You can also remove existing bricks using:

This means that you can remove a brick with smaller hard drives, upgrade the harddrives, and re-integrate into the cluster with bigger hard drives. This means that you have a cloud like storage solution which you can easily grow as necessary without worrying about resizing underlying filesystems or hotswapping hardisks or any of that hassle.

In fact, we also considered GFS2 but looking around on the net, there seemed to be a general consensus recommending ocfs2 over gfs2.

Ubuntu makes it pretty easy to install and manage ocfs2 clusters. You just need to install ocfs2-tools and ocfs2console. You can then use the console to manage the cluster.

What I totally missed in all of my research and understanding, and due to lack of in depth knowledge on clustered filesystems was that OCFS2 (and GFS2 for that matter) are shared disk file systems.

What does this mean?

Wikipedia defines a shared disk filesystem as being “shared by being simultaneously mounted on multiple servers.”

This essentially means that the storage medium is mounted on to cluster. The cluster is a collection of the clients. The storage is traditionally a SAN mount point. This means that a shared storage space is accessed at high speeds by a number of clients.

From a simplistic point of view, this is not that different from mounting the SAN point onto a server and then running an NFS server on it with all the clients mounting over NFS.

The main difference is that OCFS is distributed. There is no single point of failure (assuming that the storage medium is redundant which a SAN would be). The NFS server is a clear single point of failure.

If you do not have access to or want to use a SAN, you can also use DRBD.

NFS is described as a network filesystem and GlusterFS is described as a NAS file system.

There has been a solution for this for several years and the first time I did this, I installed cygwin with X11 but there is a far simpler way to accomplish this.

Install XMing. I then used putty, which has the forward X11 option. Once logged in, running xeyes shows the window exported onto my Windows 7. Ah.. so much better.

I actually used this to run terminator to connect to a number of servers. Over local LAN, the windows didn't have any perceptible lag or delay. It was more or less like running it locally.

It is possible to set up shortcuts to run an application through putty and have it exported to your desktop. I haven't played with this enough to comment though.

This of course only worked because I have another box which is running Linux. If that is not the case for you, then you might want to try VirtualBox but since the linux kernel developers have described the kernel modules as tainted crap, you might want to consider vmware instead which is an excellent product.

On the bright side, there are a few new things that have been worked on so hopefully plenty of things to cover soon.

We manage a number of servers and all over the place and all of them require to be backed up. We also have a number of desktops all with mirrored disks also getting backed up.

I like things to be all nicely efficient and its annoying when one server / desktop runs out of space when another two (or ten) has plenty of space. We grew to dislike NFS particularly due to the single point of failure and there were few other options.

We had tried glusterfs a few years ago (think it was at version 1.3 or something) and there were various issues particularly around small files and configuration was an absolute nightmare.

With high hopes that version 3.2 was exactly what we were looking for, we set up three basic machines for testing

Previously, glusterfs required all the configuration to be completed manually and with text files. It also required a fairly detailed knowledge of what they called translators and a lot of tweaking and fiddling with parameters.

I am very happy to report that this is no longer the case with 3.2.

The three servers(cserver[1-3]) are running Ubuntu and was updated to Natty (11.10) to get access to glusterfs 3.2 (11.04 only had 3.0). One thing to bear in mind is that the glusterfs website seemed to only have the 64 bit version but Ubuntu 11.10 also has the 32bit version.

Installing the server part of glusterfs was simple and straightfoward

Once this was done all three servers (terminator is a godsend when doing these things across a number of servers), adding the servers into a “cluster” was easy enough.

The thing to note is that these probe statements are two way. In other words, all three servers are now part of the same cluster.

The thing is that if you type in an incorrect hostname, the probe will keep trying to connect to it. I haven’t left it running long enough to know if it every returns.

Thankfully, removing host does-not-exist is simple enough

Creating a volume is straightforward. There are a number of different types of volumes which you can find out from the documentation. In this particular instance, we are creating a distributed replicated

The reason I have not included cserver3 in here is that the volume needs a multiple of the replica number of bricks. In the case, the there needs to be a muliple of 2 number of bricks.

Additionally, you could use rdma instead of tcp if you are using infiniband

Starting the volume is simple enough

this volume is now accessible from all the boxes in the cluster

Mounting this from another box is easy

If you get the error of “endpoint not connected” when listing the content of the mount, it is likely because the volume is not started.

If you are curious, check the gdata folders in the bricks after copying some files into the mount and you’ll find them show up intact and on both bricks in the above example.

Luckily, working in Linux means that I can spread each stream of work into the various workspaces.

Now GNOME 3 makes things a little more complicated with the dynamic workspaces but I’m learning to use it to my advantage

However, with Ubuntu 11.10 Oneiric Ocelot and GNOME 3, I seem to be running into an issue regularly…If I leave my computer for a while, it doesn’t unlock correctly. The screen remains black and I can’t move the mouse to my second screen and the unlock screen doesn’t show up.

Thinking about it, it seems like there might be two screen savers being started but I shall investigate that tomorrow. I have the same issue at both work and home so it is more likely to be related to Ubuntu + GNOME 3 or something about the way I set things up.

I  usually resolve this by logging into the console and here a neat trick for killing all our processes in one fell swoop.

Another thing I have been doing a bit more of recently is gaming which involves rebooting in Windows.

Both of the above leaves me with a restarted workspace. Starting up the applications pops them all into the same workspace. Chrome is especially a nightmare. I might have 135 open tabs but they are in about 6 windows spread across four workspaces.

It is annoying to have to distribute these things out each time.

After having done much research, I have not been able to find a clean automated solution.

There are two half solution that I have found however.

The first one is Devil’s Pie and for a graphical interface gdevilspie. According the website for Devil’s Pie, it is “A totally crack-ridden program for freaks and weirdos who want precise control over what windows do when they appear. If you want all XChat windows to be on desktop 3, in the lower-left, at 40% transparency, you can do it.”

Unfortunately, that is exactly what it is. If you pre-determine where you want your windows to be, you can use this very useful application. However, that is not quite what I want. I want the current configuration to be remember. Exactly like how Chromium remembers which tabs are in which order in which windows and their position on the workspace, but for multiple workspaces.

Unfortunately, I couldn’t find any way to save the current state.

There is however, another tool I found scouring the web.

libwnck-3-dev is what I installed on my Ubuntu box. There are two key commands here

This will list all the windows across all the workspaces. To get more information on a specific Window,

The XID is the number returned next to each window from the first command. The post that I  mentioned above has a nifty tool attached that saves the window positions and can also restore them using wnckprop.

However, it saves them based on the Window title. This of course doesn’t work for Chromium or such Windows that changes the title each time you change the tab.

However, if the save is the last command you run and the restore is the first command you run after opening up the windows, it can restore the windows into the correct workspaces.

With the idea of the dynamic workspaces in GNOME 3, you might have to initialise the workspaces first but it is better than spending five minutes after logging in each time re-arranging windows…

Tie this power with either grep -l. (Thanks to Steve for pointing out a mistake in the following, now corrected)

or find

If there are multiple changes you want to make, just put them all into a file and pass it in via the -f flag.

file: replacements.patterns

and the command, using find to iterate through all files in the current directory and subdirectories.

et voila - hope it helps.

I installed synergy on both ends, with the linux one as the server

and the mac as the client

http://sourceforge.net/projects/synergykm/

and it worked.

There was just one very very annoying problem. The Ctrl key and Cmd keys were different. This really messed with my muscle memory. After some hunting around, I just had to update my .synergy.conf file in linux. Here is the relevant section

et voila. It now works a charm. I  have neglected the configuration of the synergykm and synergys but these can be figured out easily ;-)

I needed host networking to be able to access network resources (Samba shares etc.) which does not work if the guest OS is on NAT :-(

Solving it was easy though… I assume Vista is installed as a guest with the guest additions and that your user account is a part of the vboxusers group.

On the linux host, first install bridge utils. I run Ubuntu, so it was as easy as:

Next, you need to set up the bridge; again, easy on Ubuntu:

add the following section to /etc/network/interfaces

Add the interfaces to VirtualBox

Within the VirtualBox Guest settings, choose Host Networking and fo the interface, choose br0

bring the interface up:

and start your guest os… et voila, it just works…

I am making some assumptions based on how I see twitter working. I have little information about how it is architected apart from some posts that suggests that it is running ruby on rails with MySQL?

Twitter is in the rare category where there is a very large number of data being added. There should be no updates (except to user information but there should be relatively very small amount of that). There is no need for transactionality. If I guess right, it should be a large amount of inserts and selects.

While a relational database is probably the only viable choice for the time being, I think that twitter can scale and perform better if all the extra bits of a relational database system was removed.

I love challenges like this. Technical ones are easier ;-)

If I didn’t have a lifetime job, I would prototype this in a bit more depth. Garry pointed me in the direction of Hadoop. Having had a quick look at it, it can take care of the infrastructure, clustering and massive horizontal scaling requirements.

Now for the data layer on top. How to store and retrieve the data. HBase is probably a good option but doing it manually should be fairly straightforward too.

From my limited understanding of twitter, there are two key pieces of functionality, the timelines and search.

The timelines can be solved by storing each tweet as a file within a directory structure. My tweets would go into

/w/o/r/d/s/o/n/s/a/n/d/<tweet-filename>

The filename would be <username>-<timestamp>

For the public timeline, you just have a similar folder structure, but with the timestamp, for example, the timestamp 1236158897 would go into the following structure as a symlink

/1/2/3/6/1/5/8/8/9/7/<username>

For search, pick up each word in the tweet and pop the tweet as a symlink into that folder. You could have a folder per word or follow the structure above.

/t/w/i/t/t/e/r/<username>-<timestamp> OR

twitter/<username>-<timestamp>

You would then have an application running on top with a distributed cache with an API to ease access into the data easier than direct file access. Running on Linux, the kernel will take care of the large part of the automatic caching and buffering as long as there is enough RAM on the box.

This can in theory be done without Hadoop in between and separating the directory structures across multiple servers but that can have complications of its own, especially with adding and removing boxes for scalability.

You are also likely to run into issues with the number of files / sub-directories limits but they can be solved by ‘archiving’ - multiple options for that too…

Thinking about this problem brought me back to the good old days of working on the search mechanism within megabus.com. We needed the site to deal with a large number of searches on limited hardware when the project was still classified as a pilot.

With some hard work and experimentation, we were able to reduce the search time to a tenth of the original time.

I’ll admit that I don’t know the details or the intricacies of the requirements that twitter has. I have probably over-simplified the problem but it was still fun to think about. If you can think of problems with this - let me know; I wanna turn them into opportunities ;-)

However, recently, I have been niggled by the amount of time it takes to switch perspectives on eclipse. It takes a good 4 seconds to switch between perspectives.There is also a noticeable lag when performing some operations.

To resolve this, I spent a lot of time looking at the linux real-time and low-latency patches. I had expected that running X11 applications remotely would not cause a bottleneck over a gigabit link. Turns out that I was wrong.

To test this, I ran a vnc server on the application server and found that switching perspectives on there was super fast.

To be able to resolve this, the first thing to do was to remove any latency put on the X->X communication by ssh.

We use gdm, so I had to enable to TCP on there first. Do this using the following config line in /etc/gdm/gdm.conf

Restart gdm

on the remote host, export DISPLAY