Custom implementation of ffmpeg on an Ubuntu 8.10 server

Update (20 october 2009) – This is now two Ubuntu releases out of date, so please check some more recent sources before trying this.

I’ve been working for a while on a problem at the NFB where we are moving a Drupal site to a new server running Ubuntu Intrepid. The site depends on transcoding handled by the outdated Drupal video module. Because the video module did only a single-pass encode, NFB staff created a shell script to fool the video module and replace the single-pass encode with a double-pass encode that normalized the film audio.

The result was a generally better quality product. Now the problem is that this shell script uses ffmpeg for multiplexing at the end of the two-pass encoding process, and that the Ubuntu Intrepid ffmpeg package crashed gloriously on every multiplex.

So, I decided to compile a new ffmpeg package from source. But my problems with multiplexing didn’t end there. Many of the blog posts I found made mention of source patches for the ubuntu x264 package. Anyway, I finally found my answer in the ubuntu forums:

http://ubuntuforums.org/showthread.php?t=786095

I followed these steps in a local virtual machine I created just for this purpose. After doing this, I had two deb packages ready (or so I thought) to be installed on our transcoding server. I fired off an e-mail to the supervisor of our IT department and transferred the files over to my home account on the server.

My supervisor, having lost custom packages to ubuntu updates in the past, insisted on a more comprehensive install, and sent me this link:

http://linux.togaware.com/survivor/Local_Debian.html

We followed these instructions to create a local repository. Then we popped into aptitude and tried to install our packages. BANG! Dependency errors.

After a couple of hours of combined troubleshooting we hit on a recipe that worked, so with no further ado, I present you with a complete recipe for building Debian packages of ffmpeg and x264 from source, transferring them to another machine, creating a local repository, and successfully installing them to your machine.

Compiling the custom packages

First we purged the existing packages:

 apt-get purge apt-get purge ffmpeg x264 libx264-dev 

Then we installed the following:

 apt-get install build-essential subversion git-core checkinstall yasm texi2html libfaac-dev libfaad-dev libmp3lame-dev libtheora-dev libxvidcore4-dev 

The build-essential package contains compilation tools, subversion and git-core are to fetch the required sources, the libs are for configuring ffmpeg, and checkinstall is the program we will use to install the programs and generate .debs.

Next we fetched, configured and compiled x264:

 git clone git://git.videolan.org/x264.git 
 cd x264 
 ./configure --enable-shared 
 make 
 checkinstall --fstrans=no --install=yes --pkgname=x264 --pkgversion "1:0.svn`date +%Y%m%d`-0.0ubuntu1" 
 ldconfig 

And we did the same for ffmpeg:

 svn checkout svn://svn.ffmpeg.org/ffmpeg/trunk ffmpeg 
 cd ffmpeg 
 ./configure --enable-gpl --enable-postproc --enable-pthreads --enable-libfaac --enable-libfaad --enable-libmp3lame --enable-libtheora --enable-libx264 --enable-libxvid 
 make 
 checkinstall --fstrans=no --install=yes --pkgname=ffmpeg --pkgversion "3:0.svn`date +%Y%m%d`-12ubuntu3" 

Modifying the control files on the transcoding server

Checkinstall does not add dependancies to the packages, so we did that manually.

First we uncompressed the .deb packages and their control file:

 dpkg-deb -x x264_1:0.svn20090206-0.0ubuntu1-1_i386.deb x264 
 dpkg-deb -x ffmpeg_3:0.svn20090206-12ubuntu3-1_i386.deb ffmpeg 
 dpkg-deb --control x264_1:0.svn20090206-0.0ubuntu1-1_i386.deb x264/DEBIAN 
 dpkg-deb --control ffmpeg_3:0.svn20090206-12ubuntu3-1_i386.deb ffmpeg/DEBIAN 

Then we edited the control files:

vim x264/DEBIAN/control
vim ffmpeg/DEBIAN/control

We copied and pasted the dependancies from

 /var/lib/apt/lists/ubuntu.nfb.ca_ubuntu_dists_intrepid_main_binary-i386_Packages 

for ffmpeg, and

 /var/lib/apt/lists/ubuntu.nfb.ca_ubuntu_dists_intrepid_multiverse_binary-i386_Packages 

for x264. We found these files by grepping for “Package: ffmpeg” or “Package: x264″, and used those search terms again within the files to locate the package information.

Here’s what our control files looked like, more or less, after this process:

Package: ffmpeg
Version: 3:0.svn20090206-12ubuntu3-1
Architecture: i386
Maintainer: root@localhost
Installed-Size: 48192
Depends: libavcodec51 (>= 3:0.svn20080206-8) | libavcodec-unstripped-51 (>= 3:0.svn20080206-8), libavdevice52 (>= 3:0.svn20080206-8) | libavdevice-unstripped-52 (>= 3:0.svn20080206-8), libavformat52 (>= 3:0.svn20080206-8) | libavformat-unstripped-52 (>= 3:0.svn20080206-8), libavutil49 (>= 3:0.svn20080206-8) | libavutil-unstripped-49 (>= 3:0.svn20080206-8), libc6 (>= 2.7), libfreetype6 (>= 2.3.5), libimlib2, libsdl1.2debian (>= 1.2.10-1), libswscale0 (>= 3:0.svn20080206-8) | libswscale-unstripped-0 (>= 3:0.svn20080206-8)
Provides: ffmpeg
Filename: dists/local/local/binary-i386/ffmpeg_3:0.svn20090206-12ubuntu3-1_i386.deb
Size: 19122026
MD5sum: 06847640d947b33b697c03182e2c593f
Section: checkinstall
Priority: extra
Description: Package created with checkinstall 1.6.1

Package: x264
Version: 1:0.svn20090206-0.0ubuntu1-1
Architecture: i386
Maintainer: root@localhost
Installed-Size: 2508
Depends: libatk1.0-0 (>= 1.20.0), libc6 (>= 2.7), libcairo2 (>= 1.6.0), libglib2.0-0 (>= 2.12.0), libgtk2.0-0 (>= 2.12.0), libpango1.0-0 (>= 1.20.2), libx11-6, libx264-59 (>= 1:0.svn20080408)
Provides: x264
Filename: dists/local/local/binary-i386/x264_1:0.svn20090206-0.0ubuntu1-1_i386.deb
Size: 892740
MD5sum: bfb45ec75936b3acdc4fc0d6be7fe5b6
Section: checkinstall
Priority: extra
Description: Package created with checkinstall 1.6.1

While we were in the control files, we also took the time to modify the maintainer and description strings. Once we were satisfied, we rebuilt our packages:

dpkg -b ffmpeg ffmpeg_3:0.svn20090206-12ubuntu3-1_i386.deb
dpkg -b x264 x264_1:0.svn20090206-0.0ubuntu1-1_i386.deb

You will be overwriting your old packages – if you want to use a different name for the rebuilt packages, then make sure to modify the “Version:” line in your control files so that the new package name and version string while match up.

Creating the local repository

We chose to create our local repository in opt:

mkdir /opt/my_repository/dists/local/local/binary-i386

And set permissions.

chmod -R g+w /opt/my_repository
chown -R root:admin /opt/my_repository

We copied both packages to inside /opt/my_repository/dists/local/local/binary-i386 and ran a command to create package information:

dpkg-scanpackages dists/local/local/binary-i386 /dev/null > dists/local/local/binary-i386/Packages

This command MUST be run from inside /opt/my_repository. Otherwise when you use apt or one of its friends to try to install the packages, they won’t be able to find them.

We added these lines to /etc/apt/sources.list:

#Repository of local packages
deb file:/opt/my_repository local local

At this point we were able to start aptitude, update (“u”) the package lists, find our custom packages and add them (“+”) and finally install them (“g”).

The final step was to create a symbolic link to ffmpeg in /usr/bin, because our shell script and drupal were looking for /usr/bin/ffmpeg.

ln -s /usr/local/bin/ffmpeg /usr/bin/ffmpeg