As a programmer, you can never, and should never, write every single line of code that your code needs[1]. It turns out that you end up writing very little code for what your programs actually do. Lets start with a simple example. Every CS student learns how to implement a linked list, but how many times does a real life programmer actually write their own linked list implementation from scratch? Never! I’m not trying to debate whether or not CS students should have to learn how to implement a linked list, which BTW I think they should. My point is that all common programming languages provide libraries for common programming tasks, like linked lists, hash maps, sorting algorithms, math functions, and so on, so you don’t have to write them from scratch.

There are also more necessary uses of libraries that offer more than just saving time and effort. This is the case when you want to interact with any external component. GUI libraries are a good example. If you want to write software for the Android platform you must use their libraries in order to display things on the screen. There are hundreds, maybe thousands, of classes written by Google that provides access to the Android platform, primarily for creating and manipulating graphical widgets.

But how do you know how to use libraries? The typical approach to convey to you what a library does and how to use it is through API documentation and code examples. This helps most of the time, but sometimes it is a struggle to use libraries correctly. Sometimes you don’t know exactly what the API classes, methods, functions or whatever expect in and sometimes you don’t know exactly what will come out, even with what would be considered high quality documentation. Sometimes it is just complicated. Then there is poor quality documentation where it becomes a guessing game.

Where does API documentation and code examples come from? Code. Why then is code generally considered a non-traditional or even unacceptable method of learning how libraries work and how to use them in your code? First, many people are very protective of their code and don’t like to let other people see it. Maybe they think they have something special and don’t want anyone stealing their ideas. Maybe they are self-conscious about their work and are afraid of criticism or that other people will discover their poor coding abilities. Second, libraries are usually complicated and without an understanding of how the code is designed it can be difficult to make use it. It is also usually more time consuming to dig through and learn someone else’s code rather than reading their API documentation or examples. But sometimes it is worth it.

The first hurdle, actually getting people to let you read their source code, isn’t something I can appropriately address in this post. But I can say that this hurdle is a non-issue with FLOSS (free (as in libre) and open source software). It is for the very point of this post that I avoid non-free software libraries like the plague.

The second hurdle, the difficulty in reading library code, is usually easier to get around. Reading code is the most descriptive way to truly understand how a library works and how to use it. With a little time and practice, it’s really not that hard.

An interesting way I often use library code is for optimizations. As a simple example, let’s say you are using a sort method. Often times libraries will offer many ways of passing data into basically the same method. For a sort method, the library may accept either a List object or an array. If I pass in a List, does the library convert the List to an array, perform the sort on the array, then turn that array back into a list? If I can work with an array in my code just as easily as a list then I can skip the extra conversion by just working with an array. The information that says the List method has to convert to and from an array could be provided in other documentation, like Javadocs, but if it wasn’t then the only way to know is to look at the code to see what it does.

I often look at JDK library code. Sometimes it is for optimizations. Sometimes I can better understand its use if I look at the code. Sometimes it helps decide if I should extend a library class, extend one of its base classes, or something else. Sometimes I am just curious. But I know that I am more effective at coding and my code is better when I have access to library source code.

Reading other peoples code can also help you be a better programmer. You can discover new and better ways of doing things and you also get better at reading other peoples code which has benefits beyond reading library code.

So even though you may never write a linked list implementation, you may some day need to read the code of an implementation you are using and should know what it’s doing and why.

[1] This entire post is with exception of extremely uncommon situations, such as embedded systems or low-level system control.

I would love to be able to follow the ideals of Free Software pioneered and maintained by Richard Stallman and the Free Software Foundation, but in a practical world I can’t commit 100%. They are great ideals for which to strive, but they are just not practical 100% of the time for 100% of the people. Free Software rights are essential for some users, they are less essential for more users and most are oblivious to them. Although Free Software rights benefit everyone, even those are don’t require or are oblivious to them. This is because everyone can reap the benefits of the few who take advantage of them.

I think I lie in the range of Free Software being “slightly less essential.” I choose to use Free Software wherever possibly, which for me is the majority of my software needs. I have acknowledged that there are some things I like to do with software that can’t, and probably will never fully, be done with Free Software. In my case the primary thing I can’t do with Free Software is gaming. Although it pains the ethics centers of my brain, I have an Xbox 360 and enjoy playing it.

Stallman/FSF seem to focus solely on the rights of the end user, and to impose those values on software developers and distributors. But I have personally come to accept that software developers and distributors have rights too, and those rights include imposing whatever rules they feel fit to impose on their users. But if software developers/distributors wish to restrict the rights of their users, those users have the right, some would say obligation, to reject the software.

My optimistic hope is that users will keep software developers/distributors in-check by calling them out and boycotting their software when unethical and overly restrictive rules are imposed. The more common and effective response of the community, however, has been to create a Free Software alternative. But we shouldn’t overlook the power of a passonate community.

I’m surprised I haven’t figured this trick out yet, but I was kindof forced to when I got a new hard drive for my laptop, didn’t want to reinstall Linux, and didn’t have a lot of options. The only place I could back it up was on my local server. A usb-attached hard drive would probably be best, but I only had a network, so I needed to get the data to my server. Rsync might have worked, but I expect it would have taken a very long time. The best option would be to transfer a gzip’ed archive but I couldn’t save it locally then scp it. So I had to direct the output of tar/gz directly to the network. I’ve done a lot of things with ssh, but not this. What I found was the way scp handles stdin and stdout.

If you pass a command as a final argument to ssh it will execute the command remotely, but pipe stdin from the local terminal to the remote command and stdout from the remote command to the local terminal. So all I had to was execute a remote command to save stdin to a file on the remote system. This can be done via the command:

$ tar -cz . | ssh user@host "cat > file.tar.gz"

The tar -cz . says “create a gzip compressed archive of the current directory.” The | (pipe) says “take the output of the tar command and pipe it to the ssh command.” The ssh user@host “cat > file.tar.gz” command says “ssh to host as user and execute the command ‘cat > file.tar.gz’”. In cat > file.tar.gz, the cat command is there to properly catch and redirect the output and says “take my stdin and output it to stdout” and the > file.tar.gz says “direct the output of cat to the file file.tar.gz”.

The tar command gets us the gzip’d archive, the ssh command lets us pipe the output to a remote command, cat gives us a command that ssh can execute that takes the command input (the output of the tar command) and output that to a file.

Then once I have the backup and setup the new hard drive in my laptop I can restore the data using the command:

$ ssh user@host "cat file.tar.gz" | tar -xz

This does the same thing in reverse. It sshs to host as user and runs the command cat file.tar.gz which reads the file.tar.gz and outputs it to stdout. Then we capture that output and pipe it through tar, locally, which gunzips/untars the data to the current directory.

I could have also mounted a remote directory on my server using something like nfs, but I didn’t feel like taking the time to set that up.

This is a really neat example of stream manipulation in Linux. Hopefully you can learn something from it.

Note that I did all this from an Ubuntu Live CD so I wasn’t actively using my old hard drive (mounted readonly with mount -o ro /dev/sda3 /mnt/sda3) when I was backing up the data and so I could setup my new hard drive. The only other thing I had to do after I restored the data was to reconfigure grub, /etc/fstab, and /etc/blkid.tab with the new UUID’s for the hard drive. I first had to use /dev/sdaX instead of UUID’s to be able to boot and find the UUID (I couldn’t find the UUID in the Ubuntu 7.10 live cd, I’m guessing because it was a little old, and I also didn’t feel like downloading and burning 8.10). Then I could configure the new UUIDs and reboot and all was good. Let me know if you would like more details on the UUID part.

I’m too lazy to backfill my old 105 posts from Blogger, so if you’re interested in seeing anything from before this post it’s under my Blogger site.

View Old Posts On Blogger

My website has entered a new phase of existence. I started out with a home-brewed blog and website, then used Blogger to power the blog part of the site, now using WordPress to power the whole thing. It was time for change and I wanted something that was easier to setup and manage and WordPress seems to fit the bill. Hope you enjoy it. Just bear with me while I backfill my old content and build up new content.