Create a custom Raspbian OS image for production.

Recently, me and my fellow team mate Tharindu Muhandiram ran into a bit of an issue with regard to building a custom image for our Platformer IoT Gateway. The issue we had was: we had a working pi with Raspbian OS installed in a memory card and we have added some additional libraries and scripts to it to get our IoT solution working. All the tests were passing and everything was ok. Here goes our issue.

In order for us rollout our IoT solution seamlessly, we had to repeat the above process manually for each device. ie, flash the base Raspbian OS and boot it up, manually add libraries and scripts we need. This is going to be a cumbersome task and we hate manual repeating tasks. To avoid this we decided to build a custom image from an already working device and flash that image to the other devices.

We went through so many tutorials, videos etc to achieve this, but none of them matched our use case. However, we managed to achieve our goal and I thought I’d put it up in a blog post so that it might help someone else save heaps of time.

Pre Requisites

You need to have a linux environment with gparted installed. If you are on a Mac or Windows, You will have to quickly download virtualbox and run an ubuntu instance.

in order to install gparted in ubuntu,

sudo apt-get install gparted

Note: We used ubuntu 18.04 LTS for this tutorial

Step 1: Clone SD Card

Okay, assuming you are logged into ubuntu and have inserted the SD card containing the custom Raspbian OS which was used to run your pi, you can open up the terminal and list the disks as follows.

sudo fdisk -l

Note: If you are using a virtualBox, Please mount the memory card inside so that you can access it from the terminal.

This was run in a virtual box, depending on your system, the path of your sd card may change. In this example, as shown in the screenshot, the path to my sd card would be /dev/sdb.

Okay, now we need to completely clone the SD Card and we do it using the following command.

sudo dd if=/dev/sdb of=/your/path/to/clone.img

In this command if stands for input file which is my disk path /dev/sdb. This value may change in yours. And of stands for output file, which you can give any path you specify.

This process will take some time as it will copy what’s in your memory card block by block. The bigger the memory card size is, longer it would take. You will see in the command line when the process is finished.

Note: In order for this to complete in a virtualBox, your free disk space should be more than atleast twice the size of your SD Card.

In our case, the size of the memory card was about 14.5GB and it took approximately 20–30 mins to fully clone to our non SSD Hard Disk.

Step 2: Shrinking the Image

Okay, once the image was cloned, you might be wondering about the size of the image. In my case the image was 14.5gb. But my custom os only takes up about 4gb as I could remember. Shrinking the image was the next problem we had to tackle. For this, Linux is your only choice as gparted is only available in linux.

To do this, we are going to use pyShrink script file written by Drewsif which can be found here.

Open up your terminal and execute the following commands line by line in your terminal.

wget  https://raw.githubusercontent.com/Drewsif/PiShrink/master/pishrink.sh
chmod +x pishrink.sh
sudo mv pishrink.sh /usr/local/bin

Okay with that done, the rest is easy. To shrink your image, run the following.

sudo pishrink.sh /your/path/to/clone.img /your/path/to/clone-shrink.img

This will also take some time and finally it will give you an output saying how much your initial image was shrunk in the new file.

That’s pretty much it, and your custom image is now ready for flashing.

Step 3: Flash the image to a new SD Card

This is relatively easy. Download the Etcher application for your OS and select your newly shrunken image, then select the new SD card and click the button Flash And you are all good.

Step 4: Compress the image further

Still with an image of 4gb+ it’s a bit too much to be shared with others on the internet. We can further compress this by running gzip.

gzip -9 /your/path/to/clone-shrink.img

This will zip your image file even further so that you can easily upload it and store in a cloud storage bucket or a drive.

Thanks for reading. I hope this helped you.
Cheers!