The simplest way to add basically any file format into iTunes is to make sure you have the developer tools installed (it comes with your Mac OS X cd) and just go to the directory of your movie/audio files and do this (in this example I’ve gone into the directory where I have a ton of MKV files):
SetFile -t “MooV” -c “TVOD” *.mkv
Then Mac OS X will identify the files as Quicktime movies. Other than that, nothing has changed, it’s still MKV files, we still need Perian installed to play the files/be able to use the codecs. Oh and if you like me have 720p MKV rips, importing the files into iTunes will take a really, really long time. Of course these files wont work on your iPod, but thats not the point here, the point is to have a tidy nice iTunes on, say your Mac Mini which acts as a HTPC, and being able to have a nice, clean Front Row experience with Movie artwork, info and so on.
Update Jan 26/09: Be sure to check out these comments below, Videodrive seems to be a good piece of software to help you out and also check out Marcus tip about using Quicktime to “save as mov” in order to get iTunes to like the mkv better.
Update Apr 25/08: Adding artwork and stuff seems to crash my iTunes. Gargh! Even though it’s slow as hell to add the mkvs and I cant really view the movies with coverflow or add artwork..its still my best bet as of today.