Showing posts with label HTPC. Show all posts
Showing posts with label HTPC. Show all posts

Saturday, February 16, 2013

MDL: Some More XBMC Tweaks

I’m now running two instances of XBMC Frodo each sharing a single MySQL database.  Frodo has allowed me to strip out some of the bandaids I put in place to make the experience as seamless as possible.
The box that I have in my family room is an Acer Aspire running Ubuntu.  That is also where MySQL is running.  The box I have in the basement is a Raspberry Pi running RaspBMC.
I used to have to store all of my thumbnails on an NFS mount and create linked directories on every box because XBMC wouldn’t regenerate thumbs locally if the content was already defined in the database.  Now with Frodo that isn’t necessary as XBMC will create thumbnails locally regardless of the existence in the database.
The other piece that I was able to strip out was a kludge of a script that attempted to force regeneration of  metadata and thumbnails when I made changes after the initial discovery.  MediaElch now allows me to edit metadata and force XBMC discovery of those change.
My tweaking is now limited to:
  • Creating symlinked NFS mounts for my content so that they show up in local paths (this is a work around so that I can use both Windows and Linux as XBMC hosts).
  • Some keymap customizations to work around limited remote functionality using Raspberry Pi and CEC.
  • Creating some dependencies for MySQL and XBMC.
Creating the database dependency in my family room was easy since the database was local.  I just needed to edit /etc/init/xbmc-live.conf and add:
start on started mysql 
pre-start script
while [ `mysqladmin ping -h 192.168.1.130 -uxbmc -pxbmc -s | grep -c alive` -eq$
do
  sleep 1
done
end script
sleep 5

Creating the dependency on the Pi was a bit more challenging.  After installing mysql-client via apt-get I added this code to the top of /scripts/xbmc-watchdog.sh.
#check to see if mySQL is running on the family room PC
while [ `mysqladmin ping -h 192.168.1.130 -uxbmc -pxbmc -s | grep -c alive` -eq 0 ]
do
  sleep 1
done

sleep 5

I’m reasonable sure that storing your password cleartext in a configuration script is dumb, but I doubt anyone will be trying to break in and sabotage my music collection.  I also don’t know if this is precisely the correct place to put the delay – but it seems to work so I’m declaring success.

edited to add pre-start script to family room config

Saturday, April 14, 2012

MDL: Maintain the XBMC Library

One of the most frustrating things about XBMC is the static nature of its library.  The forums are full of people asking “How do you force the XBMC library to refresh?”  Many of them are asking specifically about the thumbnails and fanart.  My biggest concern is the metadata.

I have a mild case of OCD and am constantly tweaking the information about my movie collection.  Unfortunately, XBMC doesn’t ever pick up this information.  You can refresh movies manually, one by one.  But that process is tedious at best.

You can also delete the database and rebuild it.  But that kind of defeats the purpose of having a library, doesn’t it?  If you go this path you lose watch history, ratings and other information.

I think that I have developed a script that accomplishes what I need.  It removes just enough information to force re-population without getting rid of the things that I care about.  I currently have it setup to run once per week, which should be more than sufficient for my needs.

The script is in PowerShell, but shouldn’t be overly difficult to rewrite in just about anything.  A couple notes:

a) I am not worried about refreshing pictures here.  Though that can be accomplished pretty easily by deleting the thumbnail cache.

b) Use at your own risk.  I am not 100% certain that this is stable in all circumstances.  I’ve run it a bunch on my setup, but make no guarantees what it will do to yours.

c) I’m pretty sure its inefficient, I am not a programmer by nature.  If you find methods to improve it, please do so and post your results back here.

# You should be able to edit here and the rest of the script should work
# With the very important exception of the section marked !!!!LOOK AT ME!!!! below.
$xbmcuserid = "user"
$xbmcpassword = "pass"
$xbmcIP = "192.168.1.2"
$xbmcPort = "8000"
$mySQLIP = "192.168.1.2"
$url = "http://" + $xbmcIP + ":" + $xbmcPort + "/jsonrpc"
$xbmcDB = "xbmc_video"

# Connect to the mySQL database
[void][System.Reflection.Assembly]::LoadWithPartialName("MySql.Data")
$XBMCConnection = New-Object MySql.Data.MySqlClient.MySqlConnection
$XBMCConnection.ConnectionString = "database=" + $xbmcDB + ";server=" + $mySQLIP + ";Persist Security Info=false;user id=" + $xbmcuserid + ";pwd=" + $xbmcpassword
$XBMCConnection.Open()

# Make a copy of the movie table
$copyCommand = $XBMCConnection.CreateCommand()
$copyCommand.CommandText = "CREATE TABLE movie_copy SELECT * FROM movie"
$copyCommand.ExecuteNonQuery()

# Clean out the movie table
$cleanCommand = $XBMCConnection.CreateCommand()
$cleanCommand.CommandText = "DELETE FROM movie WHERE idMovie > 0"
$cleanCommand.ExecuteNonQuery()

# !!!!!!!!!!!!!!LOOK AT ME!!!!!!!!!!!!!!
# You are going to need to clean out the hashes associated with the movies or the scan will just skip past everything
# You can clear out all of the hashes (movies and TVs) but that will cause everything to rescan
# I prefer to just scan the stuff that I care about YMMV
# !!!!!!!!!!!!!!LOOK AT ME!!!!!!!!!!!!!!
$cleanHashCommand = $XBMCConnection.CreateCommand()
$cleanHashCommand.CommandText = "UPDATE path` SET `strHash` = '0' WHERE strPath like '%Movies/%';"
$cleanHashCommand.ExecuteNonQuery()

# Ask XBMC to look for new videos - this will reload the NFOs into the database
$command = '{"jsonrpc": "2.0", "method": "VideoLibrary.Scan", "id": 1}'
$bytes = [System.Text.Encoding]::ASCII.GetBytes($command)
$web = [System.Net.WebRequest]::Create($url)
$web.Method = "POST"
$web.ContentLength = $bytes.Length
$web.ContentType = "application/x-www-form-urlencoded"
$stream = $web.GetRequestStream()
$stream.Write($bytes,0,$bytes.Length)
$stream.close()
$reader = New-Object System.IO.Streamreader -ArgumentList $web.GetResponse().GetResponseStream()
$reader.ReadToEnd()
$reader.Close()

# Wait for 30 minutes, this is probably me being lazy. 
# I can't think of a way to see if the scanning has been completed or not
start-sleep -s 1800

# Reset the movie ids back to what they were. 
# This is required if you a) care about Recent Movies being accurate.  b) you use sets c) probably something else too
$ResetIDCommand = $XBMCConnection.CreateCommand()
$ResetIDCommand.CommandText = "UPDATE " + $xbmcDB + ".movie_copy t1, " + $xbmcDB + ".movie t2 SET t2.idMovie=t1.idMovie WHERE t1.idFile = t2.idFile"
$ResetIDCommand.ExecuteNonQuery()

# Drop the backed up table
$DropCommand = $XBMCConnection.CreateCommand()
$DropCommand.CommandText = "DROP TABLE " + $xbmcDB + ".movie_copy"
$DropCommand.ExecuteNonQuery()

# Clean the database
# Don't know if this is necessary, but it can't hurt, right?
$command = '{"jsonrpc": "2.0", "method": "VideoLibrary.Clean", "id": 1}'
$bytes = [System.Text.Encoding]::ASCII.GetBytes($command)
$web = [System.Net.WebRequest]::Create($url)
$web.Method = "POST"
$web.ContentLength = $bytes.Length
$web.ContentType = "application/x-www-form-urlencoded"
$stream = $web.GetRequestStream()
$stream.Write($bytes,0,$bytes.Length)
$stream.close()
$reader = New-Object System.IO.Streamreader -ArgumentList $web.GetResponse().GetResponseStream()
$reader.ReadToEnd()
$reader.Close()

Thursday, October 06, 2011

MDL: XBMC Issues Pausing With Remote SMB

I love my XBMC and have had great success sharing a single datasource and database across multiple clients.
With one exception – my linux box doesn’t like it when you pause TV and movies when the video file is on a remote SMB share – whenever you try the videos stop playing.  Manageable, but it annoys my wife.
I think I finally fixed it.  I created a symbolic link on both the Windows and Linux clients so that XMBC thinks that the file is a local path.  This is necessary because all of the clients need identical paths to the video files in order to share the common database.
So far so good, I can fast forward, rewind, pause and the videos seem to handle it fine. 

Update: Nope, never really fixed it.  I ended up switching to Drobo  and sharing content with both NFS and CIFS.  Ever since I made the changed everything has worked like a charm.

Technorati Tags: ,,

Sunday, August 14, 2011

MDL: Optimizing XBMC

There are a number of tweaks that I had to make on XBMC to make it work a little better with my setup.

First – since I am using MySQL I had to add some indexes and make a small change to the MySQL server.

By default it appears that MySQL wants to do a reverse DNS lookup to all client connections.  This makes the initial connection very slow, to disable this behavior add the following to my.ini in the [mysqld] section:

skip-name-resolve

The following indexes also improve performance:

use XBMC_video;
ALTER TABLE movie ADD INDEX idMovie(idMovie);
ALTER TABLE movie ADD INDEX idFile(idFile);

Another change that is essential because of the shared MySQL database is the sharing of the thumbnail directory – otherwise the thumbnails will only appear on the XBMC client that discovers the media.  To get around this I created a symbolic link on the Linux box to the shared Thumbnail directory on Windows.

Connecting to Windows shares from Linux has caused me all sorts of issues.  I initially tried mounting shares in fstab, but if the Windows box wasn’t available during Linux boot the shares were unavailable.  I switched to automounter which resolved the issue.

There are some other SAMBA tweaks to improve performance – in theory this was supposed to prevent video from stopping if it was paused or looking at info.  So far it is only partially successful.

socket options = TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=8192

Finally, ever time I upgrade something in XBMC or a supporting component the autostart fails to actually, you know, autostart XBMC.  To fix this edit the /etc/uxlaunch/uxlaunch file with the correct home directory.

Technorati Tags: ,,

Sunday, June 26, 2011

MDL: Backing Up to Flickr Part I

For better or worse I’ve decided that I would use flickr as my photo backup solution.  For one, it’s not terribly expensive – around $30 for unlimited storage – and it makes picture sharing pretty easy.  There are privacy concerns for some, but I’m simply not sure that I care.
The downside is that there aren’t really any tools out there that automate sending the pictures to flickr.  Foldr Monitr comes close, but its buggy, isn’t updated anymore and I just don’t like the choices that it has made. 
After about two years of complaining I guess I’ll just have to handle the problem myself.  Odds are that you won’t like the choices that I’ve made, but hopefully I’ve given you enough pieces that you can make the necessary modifications without having to start from scratch the way that I did.
You are going to need a database of some kind to keep track of everything – I have a copy of MySQL running already for XBMC so that’s what I’ll use.  I’m also going to use PowerShell since that is what I know which means that I’ll have to install the ODBC driver for MySQL.
Finally, I suck as a programmer.  Don’t use any of this code with the expectations that you’ll learn how to get better at scripting.  Also, don’t point out how if I changed some bit of code around it would be SOOO MUCH BETTER!   I don’t care – if my code functions for me, inefficient or not, I am quite content.
So be warned – this thing works for me and my needs 95% of the time.  And if I run across an anomaly where it doesn’t do what is expected if x & y are true – I stop doing  both x & y at the same time.  The chances that I’ll update the script to accommodate your special circumstances are pretty damn rare.  And adding any functionality at all is all but unthinkable.
All that being said, feel free to borrow the code and adapt it to your own purposes – you just can’t take this and put it into something that you are going to sell.  And if you update the scripts for public consumption – please give me my due by linking back here.  Obviously while noting how crappy the code is and how much work you had to put into it to make it semi-functional for a normal human being.
Part II will get into the script and perhaps the database – assuming that I actually finishing writing the god damned thing.

Update: I have completely abandoned this project.  Flickr is just too much of a pain in the ass.  I am almost completely migrated to CrashPlan for offsite backups of my pictures with the plus of being able to backup non-pictures as well.

Technorati Tags: ,,,

Saturday, May 08, 2010

MDL: Script to Convert MPG to AVIs

The MPG videos that I have are not getting along with XBMC.  To get around this I need to convert them all to AVI format.

avidemux has a great command utility and PowerShell has the power to batch the process.

$avidemux = New-Object System.Diagnostics.ProcessStartInfo
$avidemux.FileName = "D:\avimux\avidemux2_cli.exe"
$avidemux.windowStyle ="Hidden"

foreach
($SourceVideoFile in (Get-ChildItem @("D:\videos\") -recurse -include @("*mpg", "*.mpeg") | sort-object name))
{      
    $SourceVideoFile.fullname
    $avidemux.arguments = "--autoindex --load ""{0}"" --save ""{1}"" --output-format AVI --quit" -f $SourceVideoFile.fullname, ($SourceVideoFile.directoryname + "\" + $SourceVideoFile.basename +".avi")
    $avidemuxProcess = [System.Diagnostics.Process]::Start($avidemux)
    $avidemuxProcess.WaitForExit()
}

All done!  The videos all converted in a couple of hours.

Technorati Tags: ,,

Saturday, April 03, 2010

MDL: How to Share Databases with Multiple XBMC HTPCs

I’ve raved about how great XBMC is already.  Just to demonstrate how truly great it is, I went out and bought a Nettop so that I could install XBMC on my primary TV.

The one problem with this setup is that each install of XBMC maintains a separate database of content.  This is problematic for a couple of reasons.

First, I have OCD really bad and it bugs me beyond belief if the pictures, posters and details associated with all of the music, movies and TV shows are out of whack.

Second, XBMC keeps track of what you have already watched and you can choose to hide that content if you wish.  This feature isn’t all that useful if it only tracks watched content on a TV by TV basis.

So what is the solution?  It turns out that XBMC can be configured to run off of MySQL.  And since MySQL is designed for use by more than one connection, more than one XBMC can be setup to use a single database!

I am well adept at Google-Fu but couldn’t find any decent documentation on how to set this up so it took me some time.  But don’t fret, my sweat is your gain.

Step 1: Download and install MySQL (default everything is fine).  I’m running Windows so downloaded the Windows Community version 5.1.45.  The instructions will reflect this setup, but if you are running linux you should be smart enough to translate from Windows to *nix anyway.

Step 2: Once MySQL is installed and running, log into the database by opening a command prompt and typing: ‘mysql –u root –p’

Step 3: Create a user xbmc with password xbmc: ‘CREATE USER 'xbmc' IDENTIFIED BY 'xbmc';’

Step 4: Create a couple databases:

“CREATE database xbmc_video”
“CREATE database xbmc_music”

Step 5: Grant the new user access to create and modify a new database:  “GRANT ALL ON *.* TO 'xbmc';”

note: yes, I am fully aware that there are more secure ways to accomplish this, but its a media database for Christ’s sake!

Step 6: Create a file called advancedsettings.xml and place it in %appdata%\xbmc\userdata\

Step 7: setup advancedsettings.xml to look just like this.  Note, if you didn’t follow my directions exactly you will need to modify the details to make your setup.

<advancedsettings>
    <videodatabase>
        <type>mysql</type>
        <host>localhost</host>
        <port>3306</port>
        <user>xbmc</user>
          <pass>xbmc</pass>
          <name>xbmc_video</name>
    </videodatabase>
    <musicdatabase>
        <type>mysql</type>
        <host>localhost</host>
        <port>3306</port>
        <user>xbmc</user>
          <pass>xbmc</pass>
          <name>xbmc_music</name>
    </musicdatabase>
</advancedsettings>

Step 8: Start XBMC and it will create the database for you. 

Step 9: Configure advancedsettings.xml on your second copy of XBMC making sure to change the ‘host’ entry to be the IP address of the computer hosting MySQL.

And there you go, a single copy of the database that can be shared by many computers running XBMC!

If you have spent a lot of time getting all of your metadata setup exactly the way that you want it, you can export the library before you start and import it when you are complete so that you don’t have to start over.

I also discovered one last benefit to sharing a DB on multiple computers.  The ‘New Content’ is synced on all of the hosts if you have that feature turned on.

One final gotcha – if you are using Windows you have to setup your content using SMB shares and not local drives.  XBMC doesn’t handle the ‘/’ characters correctly yet in MySQL.

Enjoy!  Hopefully I saved you a little time.

Technorati Tags: ,,

Sunday, February 21, 2010

MDL: Home Theater Meet PowerShell

IntroPart IPart II

As I mentioned in the last post, getting video to my wide array of end points can be a little challenging.  You may have been surprised when I said that getting video on a Zune was a challenge.

Well, it’s not that it is hard per se – after all the Zune software will go discover all that media from any Windows share just like it does for music.  The problem is that it needs to transcode all of that media before it can copy it to the Zune.

The other issue is those large video files aren’t optimized for playing on such a small screen.  It all works, but waiting for an hour or more to copy a single movie to your Zune isn’t my idea of a good time.  And I’m not always prepared enough to do it ahead of time.

So my solution was to couple HandBrake – a fantastic program for converting video files – with PowerShell.  I had to teach myself PowerShell along the way, but in the end I have a powerful script that converts all of my video files to smaller formats optimized for the Zune.  The good news for you is that the exact same process will work for iPods or any device that can play video with a couple small tweaks.

Step one is to download Handbrake and the Handbrake CLI (command line interface).  Launch Handbrake and create a profile that will convert videos into a format that your portable device can play.

There are a couple of items in this script that will need to be modified for your particular situation.

  1. Your preset – on Line 4 I’ve called mine simply ‘Zune’, change this to whatever profile you created for your device.
  2. The location of the Handbrake CLI on Line 2.
  3. The location of your source videos – you can have as many as you want, just make sure that they are separated by a comma – Line 7.
  4. The destination for your converted movies – Line 9.
   1: $HandBrake = New-Object System.Diagnostics.ProcessStartInfo



   2: $HandBrake.FileName = "C:\Program Files\Handbrake\HandBrakeCLI.exe"



   3: $HandBrake.windowStyle ="Hidden"



   4: $HandBrakeOptions = " --preset='Zune' "



   5:  



   6: foreach 



   7: ($SourceVideoFile in (Get-ChildItem @("D:\movies\", "D:\TV\") -recurse -include @("*mpg", "*.mp4", "*.mkv", "*.avi", "*.mpeg") | sort-object name)) 



   8: {



   9:     $ZuneOutputFile = "D:\Zune\" + ($SourceVideoFile |% { [IO.Path]::GetFileNameWithoutExtension($_) }) + ".mp4"



  10:     if (!(test-path $ZuneOutputFile))



  11:     {



  12:         



  13:         $HandBrake.arguments = $HandBrakeOptions + "-i " + '"' + $SourceVideoFile.fullname + '"' + " -o " + '"' + $ZuneOutputFile +'"'



  14:         $HandBrakeProcess = [System.Diagnostics.Process]::Start($HandBrake) 



  15:         $HandBrakeProcess.WaitForExit() 



  16:     }



  17: }




In short, the script:




  1. Compiles a list of all video files in the locations specified (and, conveniently, all of the subfolders too).


  2. Checks to see if the file exists in the destination.


  3. If it doesn’t already exist it converts it based on the profile that you set up.



Some items of note:  the script hides all of the conversion process so that it can be scheduled without interrupting any other activity on the computer it is running on. 



It will also take forever to run the first time since it needs to convert all of that video.  But if you schedule it on a routine basis it shouldn’t take too long.



Pretty cool, huh?



Technorati Tags: ,

My Digital Life – Part 2: Video Distribution

IntroPart 1

A brief recap: 

  1. I hate physical media.
  2. The world is moving to digital anyway.
  3. Just not fast enough for me.
  4. So I have to figure out how to make all this stuff work together since no one is doing it for me.

That wasn’t so bad.  Now ripping a crap-ton of media isn’t the most glamorous exercise in the world, but it isn’t exactly rocket science either – so I’m not going to explain how to do that.  Let’s just assume that all of my music and video is all in a central location.

Which is good, because it is all in a central location.  I have everything stored on a Windows 7 computer with Windows file shares. 

Cool, but how do I get everything to Zunes, Tivos and XBMCs?

Well, XBMC and music on the Zune are easy since they both recognize and can consume Windows file shares.  As an added bonus the XBox 360 can consume everything as well – though the experience is simply awful.  But the Tivo isn’t so straight forward and video on the Zune isn’t as easy as it should be.

First the Tivo.  Newer generation Tivo devices support multi-room content sharing (something Tivo calls “Multi-Room Viewing” (MRV)).  This along with Tivo Desktop provides most of the functionality I was looking for – but not all.

Open Source to the rescue!  Some enterprising gentlemen backwards engineered the protocol that Tivo uses for MRV and created an application called PyTivo that shares content across the network with any Tivo that supports MRV. 

PyTivo is very straightforward to install and use.  It isn’t 100% point and click, but its close and the forums are friendly and helpful.

The main advantage that PyTivo has over something like Tivo Desktop is that it will transcode content on the fly whereas Tivo Desktop requires that all of the content be stored in Tivo acceptable format.  Pytivo also can push metadata along with the video so that you can get names, descriptions, dates, etc populated which makes the viewing more seamless.

MRV isn’t the most fully functional method for consuming content, but it is acceptable.  The text only navigation is bland – especially after being spoiled with XBMC.  MRV doesn’t support subtitles either which comes up more often than I would have guessed.  But I can live with it, at least until I can get Live TV by some other method. 

Continued – Home Theater Meet PowerShell

Technorati Tags: ,,,