GigaOM Network: GigaOM | WebWorkerDaily | NewTeeVee | Earth2Tech | OStatic | jkOnTheRun | TheAppleBlog | NewTeeVee Live | Jobs | About | Advertise | Contact

iTunes database issues

Written on December 13, 2005 by Martin MC Brown and 91 people have commented

I love iTunes, and I love that it provides me with convenient access to all my music. But I’m beginning to get frustrated with some of the interesting DB issues present in the way iTunes was developed.

I have a large library of music and audiobooks running at just under 31,000 items using 205GB of storage. I have everything on here from Tomita to Eminem and Janet Jackson to Rufus Wainwright. All of the music is (currently) stored on a single volume on the main file server (which itself runs OS X Server).

iTunes is not, and never was, a music manager - it’s a player/organizer, but there are some aspects of the database that iTunes uses that make it frustrating to use.

Database size and updates

One of the things about iTunes I love is the ability to identify songs you haven’t played recently. But for this to work, the database has to be updated each time you play a song. And that’s my current gripe.

You see, to update the database with the play count and ‘last time played’ information iTunes has to save the in-memory version of the database to disk. The iTunes database is stored in the iTunes library file in the Music folder in your home folder. It’s usually many megabytes - there’s a lot of information in there - and it obviously increases in size significantly as the size of your music library incrases. For me, the iTunes database file is just shy of 70MB in size.

So, when updating the information for one song (something iTunes does each time you play a song), iTunes has to save a file that is 70MB in size. On my PowerBook, where I store the main version of my iTunes library, it takes 3-4 seconds, and sometimes 10 seconds, for iTunes to load, modify and resave this file. For those precious seconds, iTunes locks up - it still plays music, but the interface to iTunes freezes completely until the DB update is complete. Sometimes, it even locks up the rest of the OS - annoying if I happen to be typing and then have to wait a few seconds while the system sorts itself out.

Even worse, iTunes also generates a XML file version of the database each time it updates the song playcount too. For my library that means an additional 52MB, plus the overhead for generating the XML content in the first place.

To summarize, each time I play a song - any song - iTunes has to write out 122MB of data.

It begs the question of why on earth Apple chose to go with a single database for this - I can’t be unusual in using iTunes to manage a vast library of music. I can also foresee that as we use iTunes to access videos the problems are likely to increase as we put more and more pressure on iTunes to store the information.

Using a monolithic single file is just slowing the application down and it really wouldn’t be hard to split the file up into manageable chunks, or to use one of the embedded database libraries to access a more structured and easier to update version of the information. At a push, I don’t think it unreasonable to at least be allowed the option of using MySQL (an included component of OS X) to store the data, although I realize for many that would be extreme. There’s no reason thought that iTunes couldn’t support both a file and SQL db model.

Now I wouldn’t mind the use of a single file, if it wasn’t for the fact that it caused such a hiccup while using the application. I don’t want iTunes, or the system on which I’m running it, to lock up for a few seconds while it manipulates a huge file for the sake of updating a few bytes.

It just leaves a bad taste in the mouth.

Database storage

The other issue with the monolithic file style is that backups identify a change in that file as a requirement to backup. That means I get 70MB of backup each day just for listening to my music. That’s not backing up the music I listen to, just the file that holds the database that iTunes uses to store information about what I’ve listened.

To put it into perspective, for all my other material (the articles, books, code, applications, screenshots and diagrams) that I might create in a day, the average amount to backup is just a few megs.

So, just listening to content causes larger quantities of material to be backed up than the amount I generate. :(
XML

I love the fact that I can export my database and playlists into XML. It provides a great way for me to take my music and update an SQL database with the information. It also means I can easily share the data with other people.

So why does iTunes create an XML file every time it also saves the database?

Yes, I know the XML file is a backup, but why create it for every update to the database? Why not just each time you quit iTunes?

Those operations (sharing the DB data through XML) are steps I would elect to do when I needed to do them, not done automatically each time I update the database. Why isn’t the generation of an XML version of the database an operation I can switch off in the preferences?

99% of the time - particularly when simply playing music - I don’t need the XML file regenerated. For some users, they never need the XML file generated.

iTunes, however, thinks differently.

Using a database that links to music

One of the other oddities of the iTunes database is the way it refers to and finds the music you want to play. The server holding my music has changed IP address a couple of times in the years I’ve been using it, although the name of the machine and the name of the volume (imaginatively, Music) on which the files reside has not changed in that time.

If I forget to mount the shared Music volume before I start to play a tune, iTunes can lock up while it works out where the file is. For music added recently, recent versions of iTunes have fixed the ability to automatically mount the volume where the music is located. No problems there.

If the music is old though, then iTunes starts looking for the IP address (not the name) of the server on which the volume used to reside. Now I can, kind of, understand why it does this, but why does iTunes wait around for a minute expecting this machine to answer?

Ignoring this annoying aspect, if iTunes retains this specific information about where the file is located, how is it able to find the file when I mount a volume with the same name? If iTunes wants to be so specific about where the file is located, why does it ignore that information when it first looks for the file?

Since iTunes does find the music at /Volumes/Music/… why does it even bother recording the rest of the information? It obviously doesn’t need it to play the file.

Of course, there’s no way of block updating the information so that it always looks at the new location. Nor does iTunes intelligently update it’s database when it plays the file and discovers it on a different server (but same apparent volume) than that registered in the database.

Oh no. Instead, iTunes makes you wait a minute while iTunes times out looking for a server it shouldn’t be accessing any more anyway.

Summing up

None of these problems are show stoppers. They are, however, all annoying and show some annoying lack of thought that we are normally not subjected to by Apple engineers.

In some cases we’re talking about adding a option in the preferences and a simple check during the save process - that would eliminate the need to save the XML each time the DB was updated and save both disk space and CPU cycles most users just dont need.

In others, all that’s needed is a minor rethink on the way the database and the information it stores are handled. Splitting out the DB by the first letter of the album name would go a long way to alleviating the monolithic file problem.

Despite this, I still love iTunes and, frustrating though these issues are they are not going to stop me from using it.

Share This

Filed in: Commentary and Software

Trackbacks/Pingbacks (Trackback URL)

  1. Pingback by Rajblog » iTunes database


Comments RSSComments

  1. #1 Twist says:

    This is just one of the many things I hate about iTunes. For me Audion was always the perfect solution but it stopped wanting to run for longer than 15 minutes at Mac OS X 10.2 for me.

    I have an iBook and my music is stored on an external hard drive but I keep a subset of my music on my internal hard drive. I change this subset quite often and have to delete my iTunes database files and then re-add all the music otherwise it likes to do weird stuff. Audion had a great Linked Playlist feature to deal with stuff like this. It would watch a folder and update the playlist whenever changes were made to the folder.

    And then there is the large collection at home. If I use iTunes I also have to use a companion application to switch between the two libraries. I got sick and tired of it and just started using MacAmp Lite X. I might be losing some features but I also lost tons of headaches. Also the CPU load is way lower than with iTunes. It was a heavy app to begin with but it continues to become more bloated with each release.

  2. #2 Kevin Ballard says:

    Whoa, you just pointed out why iTunes likes to freeze up at the beginning of songs - it’s not the beginning of songs, it’s the end of songs!

  3. #3 Chris Holland says:

    i also wonder whether having a single huge database file that holds all your music information increases the risks of said file to get corrupted should it ever sit on bad hard drive sectors.

    Would journaling mitigate this issue?

  4. #4 Rex Roof says:

    Thanks so much for this article. I use iTunes to manage just shy of 50,000 songs, and I have all of these same issues. At least with them voiced here, perhaps someone at apple will get some impetus to change of these things.

  5. #5 Chris Holland says:

    It also occurs to me that there may very-well be a healthy market for Music power-users (which i’m not). Is it time for industrious developers to come-up with a “Pro” music management utility? Perhaps, as Twist mentioned, something like Audion
    .

  6. #6 Martin MC Brown says:

    Chris:

    Yes, a power or disk failure could potentially corrupt the file.

    Journaling will not help - journaling only journals filesystem metadata changes (ie, directory/file names and block usage) it doesn’t journal any data.

    Journaling is there to ensure the consistency of the file system data, not, as many people think, the data itself.

  7. #7 PecosBill says:

    You QUIT iTunes???????

    The only part of the disk that is journaled is the filesystem metadata — not the files

  8. #8 Jason Terhorst says:

    If the file gets corrupted, but you still have all of the music in the folders the way iTunes organizes them, simply delete those xml and database-related files, and drag the music folder into iTunes. It will rebuild the whole database, but you will lose the play counts, etc. (of course, if it’s corrupted already, that’s the least of your worries)

    I am a third-party developer who makes use of the XML file, and I would rather that Apple not change the way it works. It’s really easy for me to work with.

    I have no issues in my everyday use of iTunes. Maybe you simply have too much music. I think your complaints are unwarranted. I don’t know anyone who has that much music. Maybe if you think it’s necessary to have that much music on your computer, you should consider Audion. iTunes wasn’t intended to handle that much data. They never thought that anyone would be sticking that many songs into the app. Granted, they might still make some changes in the next version. They tend to change the way the XML data is structured in major point releases. Of course, I’m basing this on my study of the XML, and not the standard data file, which is undocumented.

  9. #9 Martin MC Brown says:

    Jason: I understand completely why they generate XML, and as I state in the article I use that XML myself, what I can’t understand is why the generation of the XML file isn’t an option that you can switch off if you don’t want it. Most iTunes users have no idea what it is, let alone why iTunes generates the file in the first place.

    As to having a large library, I’m sorry, I don’t but that argument. Apple computers have been used by music professionals for years and I don’t believe for a second that musicians don’t use iTunes to organize their music. They will have significantly larger collections than I do.

    As a developer, I would never design a system to use such a performance limiting database method, however big or small I expected a users music collection to be - it just seems sloppy.

    It seems though that the same approach is used in Aperture for metadata, judging by the reports I’ve seen of slow metadata editing, but Apple have stated they are going to fix that.

    Hopefully theu will fix iTunes soon too.

  10. #10 Jason Terhorst says:

    Someone out there was telling me that Apple in the past had made iTunes save the XML on quit, after changes, based on the data file, and that they may do that again in the future. That would probably help with the speed issue.
    I’m still suprised that the music library file is that big. After all, it’s merely a simple little text file… or binary or something.

  11. #11 Rex Roof says:

    after reading this I decided to try putting my itunes library in a memory disk. I have to say, I think it sped up the gui freezes quite a bit.

    I made a 400MB memory disk and mounted it in ~/Music/iTunes, using this script:


    mydev=`hdid -nomount ram://819200`
    newfs_hfs $mydev
    rm -rf ~/Music/iTunes
    mkdir ~/Music/iTunes
    mount -t hfs $mydev ~/Music/iTunes
    cp ~/iTunes_backup/* ~/Music/iTunes/

    I also put something in cron to periodically back this memory filesystem up.

  12. #12 Allen Blatter says:

    I am anxiously awaiting a fix to this large single file database problem. my music does not play correctly since it zaps too much cpu to recreate the database every song. So iTunes is unusable for listening to music at this point…a real pain in the arse.

  13. #13 Mario Mariani says:

    Nice article. Even though my library is 1/4 of yours I have the same problem here. So, while reading your article I had an idea, why not locking the itunes XML? Well, I did it and worked for me. It could at least reduce the gap which you’ve explained in your article by doing that.

    cheers

  14. #14 Mikoto says:

    I read this article and came to the same conclusion as well, lockign the library prevented my freezups from happening, hough I’m sure this may become an annoyance in itself in the near future.

  15. #15 Allen Blatter says:

    How do you “lock” an XML file? In Windows?

  16. #16 Chris Giddings says:

    Allen,

    Mark the file as being read only. It should provide the same effect.

    I hold roughly 76k tunes in my iTunes Library. While locking the library has the effect described here, I don’t think it should be considered as a long term solution.

    I think our friends at Apple probably just overlooked the issue under pressure with new features and the like.

    Martin, have you or anyone else placed a bug report for this with Apple yet?

    CG

  17. #17 Steve says:

    I have experienced a different problem with iTunes that led me to this article. My major problem is in editing the tags. Some of my tracks for whatever reason show up as numbers in the genre column. A little investigation showed that these tracks id3 v1 tag was correct, but the v2 tag had a number in the genre column. To make matters worse, I correct the incorrect genre in iTunes, but every time I play the file the wrong genre returns. I am going through my collection fixing tags using two different programs for windows (not my desired platform). Anyone know any different solutions?

  18. #18 Jose Manuel Araque says:

    I have more than 37,000 files in my Music collection. I know of some cludgy workarounds, i think iTunes Library Manager? allows you to load different libraries on demand, but it has to quit the restart the app everytime. This is just plain awful. I think the XML file used to be saved on Quit in previous versions of iTunes. As it stands now the performance of iTunes on my 450 Mhz Cube is atrocious. Apple should fix this, there is NO excuse… yes, I need a new computer :-), but the design of the iTunes database is fundamentally broken, it simply won’t scale. on the other hand I dont believe that the solution is MySQL, an RDBMS is not going to give you better performance just because.

  19. #19 Paul says:

    1) For Jason
    A) “I am a third-party developer who makes use of the XML file, and I would rather that Apple not change the way it works. It’s really easy for me to work with.”
    - better the easy way than the best/correct way eh? Why would you ask us all to live with these limitations just because you don’t want to have to change your code? Lazy? you hate quality products? It sound like you would love microsoft products that usually work, but not the way they could to make life easier for everyone.
    B) “I have no issues in my everyday use of iTunes. Maybe you simply have too much music. I think your complaints are unwarranted. I don’t know anyone who has that much music”
    - how can someone have too much music? You ever hear of DJs?

    Jason - thanks for posting; sorry to dump on you like this, but your answers are a little wacky.

    i think you guys are dancing around one of the main fundamental limitations of itunes - having the info (playcount, last played, etc.) stored with the library on a LAN instead of locally on the client. I have 3 computers and a server (which contains my music files) and I am forced to only use one computer to sync my IPODs so that I don’t lose the key infomation that makes Smart Playlists work! How do we get Itunes setup so that multiple computers running multiple itune clients all see the same info (Playcount, last played)? To solve that is one giant step for mankind in the right direction.

    any help would be appreciated. I am running windows xp on all three machines.

  20. #20 Amir Meshkin says:

    Hi,
    You said

    “I love the fact that I can export my database and playlists into XML. It provides a great way for me to take my music and update an SQL database with the information. It also means I can easily share the data with other people.”

    I need help doing exactly this and am willing to pay. Please get in contact with me. I just need to know the basics of how you can take xml information and insert it into a database. If you already have this code, and its in PHP, I would love to umm..have it. LOL

    Thanks

  21. #21 Helmar says:

    I was recently wondering about the ever degrading performance of itunes with my growing music collection (now about 30gb).

    Some web research turned up that itunes does write out it’s two big database files after each update to temp files (dozens of mbs) and then, if that went well, it replaces the originals with them.

    Of course, i couldn’t believe it. It would be incredibly stupid design: Writing out 100mb to the hdd just to change one integer, for example. It would completly ignore about 40 years of database research. Thousands of manyears of development. Databases were invented for uses like this. The technology is very mature. It would scale perfectly. Performance would improve a lot… I could go on and on…

    I only hope that there will be some guys at apple who will come to their senses again.

    Thank you

  22. #22 Ian Kennedy says:

    Dear Apple

    ITunes Track and Column Manipulations

    While I am enjoying iTunes good points, I find the lack of facilities to re-organise the order of tracks most frustrating.

    Today I discovered by accident, that if one exports a library list, two new columns come up (Disc Count and Track count) which are missing from the ‘view options’ in iTunes library.

    Can you explain this and have you any plans to include the columns in iTunes library any time soon, because I believe it could solve my problem of re-arranging tracks.

    Thanks - Do these comments get to Apple software developers?

    Ian Kennedy

  23. #23 Martin MC Brown says:

    Ian:

    This information is available within iTunes as part of the track and disc number; I think you’ll find it is exported to save have to parse 1of 10 output.

    In terms of reorganizing the order of tracks - why do you want to change the order of tracks from the default?

    And even if you do, why not change change the track number?

    MC

  24. #24 Chris Giddings says:

    Ian,

    I believe that iTunes uses this data when you are viewing by album to auto organize the tracks for you. For instance, if you have an album with three disks and 39 songs it will organize them in order by disk and then by song. The data on the tracks is able to be reviewed and altered in the song info window. Enjoy!

    CG

  25. #25 Amy says:

    Wow.. thanks for this article! I wondered why iTunes was using up so much frigging space! I have less than a gig of music on my actual computer, but about 5 gigs on my ipod. When I play music directly off my ipod on my laptop it seriously degrades the performace of my whole machine. Now, my laptop is a sad slow thing, but just listening to music shouldn’t cause such a problem!

  26. #26 Daniel Walker says:

    Hey there

    I thought this was just my PC playing up. It’s reassuring to hear that it is in fact iTunes causing the problems, and locking the XML file on my 7500 item library has made everything much more responsive - indeed, the crossfade feature actually works now. I don’t use the XML file, so this has no drawbacks for me (as far as I can tell, anyway).

    Would be nice if they fixed it. But I’m happy nonetheless! :)

  27. #27 Louis says:

    I recently started using OS X as my main desktop and was surprised at how slow itunes was at indexing my music files (30 Gb or so, sitting on a file server).
    I’m still looking for something to replace musikcube (www.musikcube.com) which I used on windows XP. It uses an sqlite db and it runs circles around itunes. Doesn’t have «teh Shiny» though…

  28. #28 John C. says:

    I’ve read many of the replies here and offer as an outsider this commentary:

    Let’s start with the fact that I’m one of those blasted Windows users. Sorry, I’ll seek forgiveness in my next life.

    For reasons of historic legacy, I use MusicMatch to catalog and manage approx. 400GB of music. In general I order my library by Genre, Artist, Date of issue, Album, Track. I care not a squat about internal rating systems. I know what I like and don’t need data to tell me. (Frankly I see ratings as a keen tool for web sites to offer collaborative filter recommendations to sell more products.)

    I use some play lists for just those things that are too hard to compile individually without a lot of data viewing.

    MusicMatch isn’t the best database in the world but it is pretty good for organizing large amounts of information.

    This weekend I bought my first iPod and am generally excited with the direction Apple is taking this industry but its clear their design is more geared for my kids than me. I installed iTunes and as an organization tool, it falls very short.

    My current dilemma is that MusicMatch and iTunes are a mirror of classic Windows v.s. Mac conflicts. Neither is happy with the other on the same computer. If MusicMatch can’t get their iPod sync painless, I may have to use both – MusicMatch to organize music and iTunes to transport content to my iPod. What a hassle.

    MusicMatch has no picture or video capability. iTunes has no large archival organizational tools. Go figure.

    I’m hoping one or the other of these companies recognizes collectors as a real market and designs appropriate software to handle this. I had hopes when Apple released their new iPod home stereo but until iPods hold a terabyte and iTunes becomes a good tool, I’ll have to continue to pipe music thru MusicMatch, to my home stereo and keep a notebook in my living room.

  29. #29 NorskVik says:

    I just lost my 3500 plus songs and 25 podcasts from my itunes. I owe it to what I read in this blog and the comments in here. I was able to recover all my songs, videos, and podcast subscriptions.

    I thank you all.

    Keith

  30. #30 Csaba says:

    Thanks for all the great posts. 25K songs and my problem has been touched on in a few posts above, but solutions seem kludgy. I’m running on XP (company owned laptop;-(

    All my songs (100GB) are on an external hard drive, but I want a subset on my laptop’s internal hard drive (I have space for about 10-20GB of music) which I travel with. How do I maintain that subset without going through too much hassle?

    The ideal solution for me would be that when the external hard drive is not plugged in, iTunes knows this and only lists the songs available on the internal drive (and not a !) Maybe the way is to have the option to have iTunes point to a different library AND updates the list in the iTunes windows. Or maybe I’m missing something obvious to others.

    I’m slowly paring through my entire collection to choose the songs I want on the internal drive, but I want to keep the entire collection intact for now.

    Thanks for any help.

    Csaba

  31. #31 Tricia says:

    Re: Itunes/database situation.
    I have about 30 gigs of music which i finally just moved to the my external drive. how does one keep one’s favorite records on the hard drive? do you bascially have to drag them in one by one?
    T

  32. #32 Peter Deslandes says:

    Hi all

    Just wanted to say a HUGE thanks to those of you who provided the suggestion of making the XML file read-only.

    I thought that iTunes slowed down for me since version 6 came out, which is maybe a contributing factor depending on the changes they implemented, but it may just be that as I was steadily ripping more and more of my CDs I hit some magic number where the quantity of information has become too much. Either way, I just made the XML file read-only and I can now listen to my music without it bringing the whole computer to a standstill at the start, a few seconds in and at the end of every track.

    FYI, I have a mid-to-high spec PC running XP and 6000 files in iTunes (so really not that many compared to some of you).

    Thanks again - I am SOOOOO happy :)
    PeterD

  33. #33 Dave says:

    Guys, fab postings on this. I ave had identical performance issue with itunes on 8000/45GB of songs. However, the reason for mys post si a desperate plea for HELP! Today I mistakenly have overwritten my iTunes DB & XML files with new empty version (thankyou itunes). I did this whilst messing around with the location of “my music” in XP! I’ve still got my 45Gb /800 songs physically on my HDD but have lost all my songs.
    I dare not plug my ipod in as I fear it would zap that too! (Currently it has all my music on it).

    Does anyone out there know if there’s a painless way for me to recover my old iTunes catalogue (which is/was about 10MB in size)?

    Many thanks in advance if there’s a helping hand out there somewhere!? :(
    Dave

  34. #34 Jose M Araque says:

    Just re-import all the files to the Library by using the “Add from Folder” or “Add File” commands under the File menu. Technically you haven’t lost the tags because the MP3s themselves (or AAC) have the tags embedded. You will lose your playlists though (if you didn’t back them up with the “Export Song List” command).

  35. #35 Dave says:

    I will have a go. Does this just re-import the catalog not the physical song files? Wil lit re-import my purchased music also? Thanks again for your much appreciated help to a newbie iTune-er. :)

  36. #36 Peter Deslandes says:

    Hi Dave

    It all depends on where your files are and how you have asked iTunes to manage your library.

    If you let iTunes copy files to the “iTunes Music folder location” (the Preferences…, Advanced, General, “Copy files…” box would be ticked for this to happen and the folder is specifed on the same preferences tab) then all you do is go to File, Add folder to library… and find the folder specified as the folder location - it will add all the files to the library database leaving them all in place. You will not have certain data like ratings or last played and as Jose said you will lose your playlists but you could always reconstruct them by referencing your iPod I guess. Bear in mind that when you reconnect your iPod you will lose the playlists on it unless you set them up in iTunes. Your purchased music will be in the same place so it will be added, although you may find you need to reauthorise those tracks in order to play them - I’m not sure of the mechanics around authorising but at least it’s quick and easy.

    Regarding your iPod, obviously what ever you do don’t reconnect until you have your library back in place in iTunes. Even then you should be cautious as I’m not sure if it will be able to match up the songs on the iPod and the PC. As such, make absolutely sure that the number of songs in iTunes matches the number on your iPod before plugging it in as it may well wipe your iPod and reload your entire library even though the tracks are the same because the database might not work from filenames. I’m sure in the past I’ve had something like this happen and it certainly reloaded a certain number of files even if not all of them.

    Hope this helps. Add another comment if not :)
    pd

  37. #37 theShephard says:

    I’m having the same frustrating problem - 21000 songs, just under 100gb of music, and I’m watching that damn xml file creep up to 18MB. I have three files, the itl at 16MB, the XML at 18MB, and the temp at 16MB.

    I think the idea of caching an xml data file in memory is a fantastic idea for small datasets (sub 1MB). It’s fast, light, and more efficient than using a rdbms.

    For music libraries though, you need a database, whether it’s an external service (mysql) or something like access mdb or Sql Server Express.

    The best music MANAGER I’ve found is mediamonkey - it handles large libraries easily, using a pretty common sense approach to indexing, etc. This might not help the mac os folks, but it’s the only heavyweight program I’ve seen out there.

    Hopefully iTunes will address their issues, ’cause it’s a fun way to buy music and I really love the podcasts.

  38. #38 Dave says:

    Peter & Jose, Thanks very much for your help. Over the w/e I re-built my itunes database and also locked the XML file so it is now read-only. (The XML locking certainly stops the crazy disk access during music play, speeding up itunes and making listening once again a sane experience). I’ve yet to plug my ipod back into my PC as itunes now thinks ALL of my music has been added as of this w/e; meaning itunes will attempt to download EVERY SONG back to my ipod I reckon.

    Does anyone know if there’s a way of refreshng/synching the ipod catalogue to itunes without having to re-download the physical songs to it?
    Otherwise I’m in for a lo-o-ng download of c45GB..

    PS: you’d think it would be possible to refresh itunes from your own ipod DB wouldn’t ya? But no - buried in the ipod user guide is a statement that you can’t update songs in itunes from yer ipod… Nice 1 Apple…not. :(

  39. #39 Peter Deslandes says:

    Hi Dave. Glad you got your library back online in iTunes. I’m sure there is software out there that will allow you to restore your library from your iPod, but as I understand it this would only do the same thing in reverse that is causing your problem, i.e. it would need to move all the music off your iPod. I don’t think there is software that deals with your specific problem where you have all the music still in both places but one database has gone wrong.

    I’m afraid you may have to just plug it in one evening and hope it has finished by the time you wake up :)

    Peter

  40. #40 Dave says:

    Right then! I plugged my ipod into my PC last night to downlaod my newly rebuild itunes DB & songs. As expected the ipod recognised that a new itunes DB now existed and needed to be re-downloaded, thereby wiping off the old ipod DB song files. I set this off, merrily re-downloading all 7920 songs. At about 3 hrs into the process my PC crashed. :(
    So now my ipod THINKS it is synchronised with itunes as the catalogue/DB says it has all 7920 tunes but it has only some of the actual physical song files from the PC. No matter what I seem to try my ipod just won’t pick-up my songs from itunes following that interrupted 1st download.

    Is there a way that I can FORCE my ipod to be refreshed with every song from itunes again? Do I need to reformat my ipod and start over? (If so will I lose licences/auths etc…?)
    Do I need to start over with yet another new itunes DB instance (to fool my ipod into thinking it needs to download all songs again?)

    The plot thickens….

  41. #41 Peter Deslandes says:

    Sorry Dave but I think you will have to start again :(

    As you suggest, you should probably do a restore of the iPod first just to make sure it’s totally clean. You won’t have any problems with licencing/authorising tracks as far as I’m aware - it’s only your PC that needs to do that.

    If you are worried that your PC will crash just because it has too much to do in one go, maybe create some playlists covering all tracks and set it to synchronise only a few at a time. Just a thought…

    Best of luck :)

  42. #42 Dave says:

    Thanks for your quick reply Peter. I will hard reset the ipod and download from itunes again. Thanks for your help again. PS: My PC freezes/crashes randomly - unrelated to itunes or volume related stuff as I think it maybe a hardware fault … :\

  43. #43 AdamM says:

    Thanks for the information. I was considering buying a huge 500GB external hard drive, storing all of my music on it, and accessing this big database through iTunes. After reading these posts I think I have decided against it.

    I have a huge MP3 collection I have collected over the years and right now it is all stored on hundreds of relatively well organized mp3 CDs (hard drives weren’t that big when I started my collection). I am a relatively recent mac convert and love how iTunes manages my music. It was tempting to want to get a big single drive to manage all of my music in one place, but I don’t want to bog down my performance too much.

    For now I’ll settle for managing a select chunk of my music on my CPU’s 80 GB hard drive. Maybe I’ll look into other options in a few years when the software for managing huge libraries is better and the hardware is even better/cheaper.

    If anyone else has any other opinions or recommendations for me, I am listening. Thanks again!

  44. #44 Engineering Dynamics says:

    Beware some 500GB hard drives have a very slow transfer rate. We purchased one recently to try it out. and found the transfer rate to be 45mb/sec. that is slower by a large margin of USB 2.0.

    We returned the big 500GB disk and purchased a fast 300GB instead.

    Engineering Dynamics

  45. #45 Lynx says:

    Is it possible to export the information in my iTunes library to other applications (ie Access, Excel)? I would like to be able to go to a database and see a listing of my ablbums along with the individuals tracks on them. A good one for me would have a tree structure (just like a directory tree in Windows Explorer) where the highest level (level 1) would be the “Artist Name”. Level 2 would be the list of all the albums of that particular artist. Each “level 2″ item would have its listing of “Level 3″, which would be the list of the songs on that album.
    In other words, when you expand an “Artist Name” you would find the list containing all the “Albums”. Expanding any “Album” would give you the list containing all the “Songs” on that album.
    iTunes has done a fantastic jobs finding the track names on my CD’s so I used it to rip CD’s. I would like a way (other than iTunes) to quickly see what I have.
    Thank you very much

    PS. This is my first time visiting this blog. You have done a FANTASTIC job putting very useful information out there. THank You All

  46. #46 theWebster says:

    Thank you all for a wonderful discussion. I too vote for mysql as a solution, it seems to work very well for most everything else I do as a web developer. I have a collection of 8k songs, 50gb, kept ob a network machine that has an upgraded cpu and some new sata drives, and is a bit flaky when it comes to stability sometimes.

    I periodically drag the library over to other machines on the network (my wife’s and kid’s and my laptop) and particularly on the older ones am always displeased about how incredibly long it takes to update — so after much of an hour of reading this blog I was inspired to try a new fix for what has become an increasingly frequent problem: freeze, crash, and renaming of my old database as (damaged) when I manage to resurrect the box. My laptop had an almost-current library on it, so I dragged just the DB and XML files over and BINGO, no hours of waiting and praying to see if the rebuild would succeed this time.

    Thank you thank you, for the forum, the answers and the pressure to Apple to make some necessary tweaks to the digital hub concept.

  47. #47 ray says:

    My computer is almost completely out of disc space so I’ve copied the whole itunes file to an external hard drive, but it seems to need an application that is still on my C drive in order to play. (I haven’t yet deleted the original itunes file on the C drive.)
    I’m not sure now what I do next.
    Very grateful for any help with this.

  48. #48 Nick says:

    Thanks for all the comments here. Its given me an insight into many problems I’ve encountered in recent times. I am running a Laptop with XP. I used to use Musicmatch before I bought an iPod and never encountered the sorts of problems outlined here. My library has increased dramatically in recent times. The management of a very large collection has been highlighted here and I was unaware of the constant writing to the XML file and wondered why iTunes kept freezing so thanks for the heads up. The only time I’ve had to worry about the XML was when I changed the location in iTunes. Luckily I had read to change the location in iTunes>close the app>move the music (to ext HDD) then reopen iTunes and voila, all your playlists, counts etc are there. I just need to remain constantly aware that unless the ext drive is mounted properly before opening itunes it reverts to the default location and it cant find your music. A right pain in the posterior…
    Anyway, I was very happy to expand my library but to my dismay found that I had many thousands of duplicate files….easy fixed???……….No…. I tried some third party software, Duplicate finder, Fixtunes. These sort of worked ok for what they were meant to do but I ended up with lots of exclamation marks in itunes (and it was more labour intensive than I expected). Enter iTunes Library Updater which had some powerful features that I was a bit scared to use.
    In the end the only way to be sure that the appropriate duplicates and other miscellaneous crap were removed and that iTunes was aware of the changes was to manually go through every track that iTunes identified as a duplicate and remove. This literally took days and I think I need new glasses now….For all of you that have encountered this, wouldnt it be great if iTunes would include a tool that removed duplicate files that had the same album name and give you the option of keeping the higher bitrate or lower or whatever?
    Anyway the other problem I have discovered is that once an mp3 file has resided in an iTunes “organised” library many of them are somehow “corrupted”.
    I’ll give you an example.
    In iTunes library I have an album by Groove Armada called “Another late night”. Album and artist info is there but there are no track names only “Track01, Track02″ etc. I copied it to another folder, removed the read only attributes and added it to Musicmatch where it appeared as “unknown” with only track numbers and no other info. (this despite the tag in iTunes obviously having album and artist info)
    Musicmatch has a facility called Super track tagging which allows you to lookup track info and album art etc. (Presumably this is from gracenote CDDB). I looked up info and Musicmatch found all the actual track names and album art, year of release etc. When I tried to update this info Musicmatch informs me it cannot edit track info as the file is corrupted. I’m positive this is an itunes related phenomena. I know itunes stores info about how many times files are copied (cannot burn..error 4450) so what other stuff do they do to the tags… All I know is that I never had any problem editing tags in Musicmatch before. I dont mind itunes but its a shame they will only use Gracenote when you insert a CD with no option to look up music already in the iTunes library. I understand why but don’t like it.
    Sure I can look up the info manually and type in the info for each track but if you have 1000’s that need editing it is a huge problem… Basically I have come to the conclusion that once MP3 tags have been touched by iTunes they are buggered for use in anything other than iTunes. I’ve checked this further by adding to Musicmatch files that had previously been in the Musicmatch library (but now reside in an iTunes “organised” folder) and about half of them appear lumped into a folder called “unknown” in the Musicmatch library interface. (copying them out of iTunes folder, removing the read only attribute and adding from the new location makes no difference).Anyone know of a pain free way to edit tags in iTunes (or elsewhere) using the online database that also updates the XML file and puts the file where it belongs according to iTunes organisational structure?

  49. #49 Nick says:

    Well I’m sorry but after my long entry I discovered there was an update for Musicmatch, which i downloaded and tried editing tags again……ooops….no more problems. It seems to have fixed the problem whatever it was. I’d still like to know if theres a way to do this tagging/lookup thing for songs already in the iTunes library.

  50. #50 Franklin Vega says:

    I have been a Mac user since almost the beginning, at least the little mac you could run a floppy disc, Mac has always been innovative, but now I see that one company cannot do everything because I see Multimedia players that are superior to the ipod and capacity exceeding Apple’s ipod, I had the 1st generation ipod now, I have the forth and it has not been worth the money spent as these little guys are made of glass crystal and one wrong drop that’s it for your ipod and trying to open it up to replace the battery, that can’t be done, I have to replace the ipod every 2 years.
    Because something is always going wrong with them, the wheel get stuck or having in a rubber case gets condensation in the ipod, the 1st generation was the best as far as reliability ( I dropped it on a concrete floor.)as just the battery would just die and you could always use a charger.
    I download itunes videos and music and when I want to listen to these songs in my car I cannot burn a cd or play a video on my screen in the car as it won’t let me copy the material, I am glad that I got the $50, itunes card for opening up an account at my bank or I would be really mad and I have the latest operating system tiger but the videos still jump around like it does not have enough memory to operate the video as I upgraded up to 1 gb of Memory the max for my iMac.
    I probably start using my Mac to surf the web and transfer my music to a pc or storage device because I want to eventually have a large library and play it in Real Player as the design effects have been awesome.
    I hope Apple gets there business together because Apple has been the leader in Technical advances, don’t fail us now, I realize you want to make money but not at the expense of poor quality products that fall apart or are breakable to buy more, be a company that want’s there customers to come back because of the new advances, people will always get the latest when they can afford the best, that is no way to build new mac customer confidence level.

  51. #51 grapester says:

    adam m… my thoughts exactly… was about to purchase a 500gb for the same purpose and have an itunes interface in the living room to browse our 4 network computers which is in the terabytes. but most of my stuff now is archived on dvd and not hard drive… finding this article makes me wonder though how this is going to handle this stuff… not good. maybe will hve to look for a different player, its a shame really if itunes cant handle big collections, which is what somebody told me, then finding this article… b/c the interface is great.

    to jason terhost you have absolutely no idea of the userbase, there are thousands and thousands of ppl out there with huge collections into the 100s of gb of music, trust me. just go on any decent network (dc ) and this is easy to see, this isn’t a recent phenomenon at all either.

  52. #52 Julio says:

    Hi everyone,

    I was having the exact same problem as everyone here was having; mainly, i was getting about a 1 to 3 second pause everytime i skipped songs or started playing a song while the database was being written. I tried uninstalling and installing an older version of itunes to see if that helped and it didn’t. Eventually what I did was looked through my computer to find any itunes related files and manually deleted all of them, including the database as well as anything within the itunes folder (After uninstalling everything first). I guess what this did was wipe every trace off my computer (which, i’m assuming, is close to what a registry wipe does). Anyhow, after this procedure I installed the newest version of itunes, the problem went away and itunes works flawlessly. I have Windows XP installed.

    I hope that helps someone. I was truly frustrated for a while.

  53. #53 kelly says:

    My problem with Itunes is this: When I go to the music store and I use the search to try to find a song, I get a pop up box that says “iTunes has encountered a problem and needs to close - Tell Microsoft about this problem. Can anyone tell me what the problem is? I have reinstalled iTunes a couple of times and have downloaded the latest version 6.0. When I first began using iTunes, I did not have this problem. It happened all of a sudden.

  54. #54 Jon G says:

    IPod Copy from http://www.wideanglesoftware.com let’s you upload all songs and playlists from any ipod into itunes. The manual sync mode in itunes lets you select the portion of the itunes library to download, rather than doing a complete sync. Therefore, you can have a huge library in itunes, and download a portion of it selectively to an ipod. I use this to (1) have different downloads on a nano and an old (larger) classic. Also I use the itunes as my music library/jukebox into my stereo, and have much more than I could ever put on a single ipod (today).

  55. #55 kelly says:

    I did find a solution to my problem that I posted on July 30 about “itunes has encountered a problem…”. All I had to do was uninstall and reinstall Quick Time Player. I use Windows and not too computer savy - maybe this could help someonelse that is also computer illiterate.

  56. #56 Jim says:

    I found this site after googling a problem I have. I am storing my iTunes music on a PC downstairs, connected via Ethernet to my Airport Extreme Base Station. I then use my MacBook Pro and its front row remote to play music on a small upstairs stereo with an Airport Express Base Station.

    I use this method because my MacBook Pro will not store my entire music collection once it is all imported. Currently, I have my music on a separate hard drive in the PC (not an external.) I have about 7800 songs, taking up around 47GB of space. My current problem is that whenever I try to burn a playlist to CD, the iTunes display says “Checking Playlist”, the progress bar completes, and then iTunes locks up tighter than a drum.

    I then logged off and tried this with my wife’s account on the PC. Her iTunes music library is in C:\Documents & Settings\%USERNAME%\My Documents\My Music\iTunes (the default location) . I can burn playlists just fine from there.

    Then, I created a new account on the PC and dragged my entire music collection from the 2nd drive into iTunes and set up the drive (M:) as my iTunes music library location. That worked fine.

    Can any one determine what this might be? I saw a post online recommending that problems like this can be solved by looking for “corrupt” music. How can I do this? Even if I create a one song playlist, iTunes still freezes when checking the Playlist.

    Thanks for any input!

    Jim

  57. #57 quaeler says:

    i wish i had this problem - i have two separate iTunes installations (6.0.5) each under osX (10.4.7), neither of which will write xml files nor will either File->Export Library… correctly (i specify a save file, it begins to write a file, and then removes that file (presumably, iTunes is erring and in catching the error is removing the output file it started as part of a cleanup)).

    has anyone seen a similar problem, or solutions to such?

    thanks
    loki

  58. #58 Peter Deslandes says:

    That’s a strange one, but I wonder if you have the same catalogue of music in each case. Maybe there is something about one or more files, e.g. the tags or the content of the tags such as odd characters or some sort of corruption, that is making the XML file screw up. Certainly to have two separate instances of iTunes running on separate machines suggests that something like that is upsetting it.

    If they are the same libraries, clear one out to check that the basic installation is not at fault and then add the music to the library in chunks until you see it go wrong.

    As it happens, I have recently bought a SqueezeBox which is a media streaming device (which is amazing, by the way!). It relies on the XML file to synchronise itself with my library in iTunes and it has to rescan if I make changes in iTunes (adding new music or whatever). To allow this I now have to unlock the XML file, let iTunes update it, and then lock it again. Nothing’s ever simple, is it?!!

    Let us know how you get on - you might find something interesting.

    best of luck

  59. #59 quaeler says:

    ya - i’ve got some mp3s that have double byte nonsense going on in the id3 tags, as well as id3 tags and filenames with latin characters with diacritics..

    still i’d have thought that such scenarios are obvious use case stuff for iTunes.. i was dreading the hunting and pecking of re-adding all the files to detect the suspicious one (and losing metadata in the process as well). do you know this: if i have a library of X files, and in iTunes, i select Y of them & deselct them (so they no longer have their checkboxes ticked), if i then File->Export Library… does it only export the (X-Y) files, or all X files? (it would make the detective work a lot nicer were it the former and not the latter)

    thanks for the feedback

  60. #60 Peter Deslandes says:

    I worked on a polish mobile project once that suffered from problems with diacritics. I agree that you would expect iTunes to cope but the mobile networks in Poland can’t always handle them so it’s not entirely surprising. I know that I have an ø in my database, and possibly more characters like it, but that’s not entirely conclusive so we are probably looking in the right area…

    I’m afraid I have never played with the export feature so you are going to have to play around with that yourself. On XP you can set up different iTunes libraries on different user accounts (although they cannot all be logged in at the same time) so if you can do something like that on your Mac you could perhaps experiment a little without messing up your current libraries?

    Hope this helps
    :)

  61. #61 Haley says:

    I am having a problem! I have two movies, the kingdom hearts 2 intro and the kingdom hearts final mix(if anyone knows what they are) videos on iTunes. One is about five minutes and the other is about 3 minutes and one is 30.0 MB and the other is 35.3 MB. When I try to download them to my ipod, it says that they wont work for my type of ipod. I have a fifth generation ipod and they are in MPEG-4 Movie format and I have over 26 GB left in space. I still am not seeing the problem and how to fix it. I spent HOURS trying to find out what was wrong! I would be HIGHLY grateful for any solution to my problem!

  62. #62 Pilya says:

    Martin,

    Although I have not had the chance to read everyone’s response. I can echo your complaints and concerns to the tee. I have a 750gb collection of 47,000 songs that i have ripped from my CDs, a big portion of which have been done in ALAC.

    My MacBook which is at the base factory specs (ie. 512mb RAM) is unable to run iTunes when I have a full library. iTunes freezes at startup… and is frozen indefinitely. While this issue is not reproducible on my PowerBook (also with 512mb), there is a considerable slow down with the large libraries. (my data/xml files are 50mb in size each). This problem however is not exclusive to any one piece of software or platform. Working in the windows world actually gets even worst. Text files (ie. XML files) are NOT appropriate above certain sizes… my experience is that they should be kept below 10mb. EVERY developer should understand that parsing through a text file is NOT efficient and should be avoided at all costs. This is just basic and simple developer knowledge.

    For all those people who truly want to understand how horrible the situation is; simply take a 50mb text file and open it up in any text reader or editor. There are very few editors out there who can handle just the opening of such files let alone the manipulation of them.

    Keep in mind that when iTunes was first introduced… and up until the release of tiger, Core Data was not around. Now that it is in the OS I can only imagine (and hope) that Apple will begin to use it for the iTunes library. This will also make life MUCH easier for developers who wish to create applications that work with iTunes.

    I have to add as well that I was a “Metadata Guru” during my tenure at MS. The “Metadata Gurus” were an exclusive group of experts who helped advise the WMP team on handling media in regards to library specific roles and metadata tagging of media. MS made every attempt to bring WMP into a SQL database as a result of the very problems that you have mentioned. As I am sure you and any other Power-user knows, no player can properly handle large collections… not the kind of collections that we have.

    I would also like to add how imperative it is that the media creation industry and various software alliances need to ditch the ID3 tagging system. One way to allow all software to improve upon the way it deals with data is to implement metadata that is agnostic to type and classification. The perfect example is to look at the differences between the metadata that the classical genre has in comparison to rock & pop. iTunes/Apple should really step up to the plate and provide an open standard that goes beyond the garage-controlled realm of ID3 and gives developers a chance to really spread their wings in regards to how a user interacts with their media libraries. In the near future this will also be important as the video, TV, and film distribution system enters the digital age. Shoe-horning an ID3 based tagging system into a TV show video clip will not suffice.

    To summarize, the problems that you see in iTunes are not exclusive to iTunes. They are through-out all software of this classification. Though it is IMHO that Apple needs to take it upon themselves as the industry leader to solve the base issues that are ever so abundant. Just moving the data into a Coredata will not solve all issues though is a good first step.

    -Michael

  63. #63 ALL2RAIN says:

    Have a client who asked me this. Trying to help.
    I have about 2,000 songs in my iTunes and have loaded them onto my iPod. They are eating up a lot of my small hard drive (only 30 gig) on my laptop. I want to remove titles off of iTune and add more and not have it take the removed ones off the iPod when I sync them the next time.
    Any help would be greatly appreciated.

  64. #64 Billie says:

    My sister downloaded the new iTunes for me without me knowing and personaly i think the older one is better. is there any way of gettin the older one back?

  65. #65 Evan says:

    So is there a music player that can deal with large collections?

    e.

  66. #66 Victor says:

    I just want to say a BIG thanks to everyone who suggested locking the XML library file to stop iTunes from locking up the OS at the beginning/end of every track.

    I’ve just spent a sunday afternoon uninstalling and reinstalling iTunes, re-building library files, updating chipset drivers, updating the BIOS, testing with and without anti-virus … even considering a full format … all in an effort to have iTunes play music _without_ freezing up. Who would’ve thought the solution would be as easy as setting an attribute on a single file!

    Thank god for Google and you guys :)

  67. #67 Alpha says:

    Thanks for the well presented information, I tried to find the reason of my problems for more than two months without success until I discovered your blog. I have about 9000 songs in my iTunes library and all the painful issues that come with them.

    I really hope Apple will fix the way the player handles the data, until then the proposed solution with locking the XML will certainly ease my troubles!

  68. #68 Nii says:

    i work with about 60,000 songs, my solution (for macs) was to create multiple user accounts, (in this case 4) and enable fast user switching which virtually allowed to have 4 differnt itunes opened, thus 4 smaller databases instead of the combined 160mb (database and xml file) i had under one itune program running. hope this helps, helped me a great deal

  69. #69 rob says:

    My iTunes won’t let me even launch the program if the library is read only.. it’s really frustrating!

  70. #70 James says:

    Hi,
    Can anyone tell me what an XML file is? Where is it located? If I’m understanding this,it is best to turn off the write privileges to this file Yes? Thanks

  71. #71 Ideasculptor says:

    This problem has been driving me nuts, too, since I exclusively use various laptops with fairly slow drives. My music library lives on an external 2.5″ disk and it has gotten to the point where I cannot use iTunes at all because I can’t get any work done. I was already having issues with iTunes because it just lacks features that people with huge libraries seem to want to use. I don’t know about you folks, but I seem to listen to my 20,000 track library in a different manner than those folks with a thousand or two tracks. I’m not going to detail my gripes here, since I have been slowly but surely working on an application that provides a nice user interface that works well on very large music libraries. I’ll post a note here if I ever get the thing into a state worthy of publishing the source. But it definitely uses SQL to maintain the backend and should be able to, fairly simply, move from SQLlite to mysql to postgres and others. I’ve tested against those 3 with no difficulties. And yes, performance when updating a single field is far superior to writing out a couple hundred megs to the disk!!!

  72. #72 Jonas Hellesøe Nielsen says:

    -> James:
    An XML file is an universal way to store information, like databases and so on.
    As http://en.wikipedia.org/wiki/XML says:

    “Its primary purpose is to facilitate the sharing of data across different information systems, particularly systems connected via the Internet.”

    See there for more.

    Lucky me, i only have 7,5K songs :D

  73. #73 James says:

    Jonas,
    Thanks for the explanation. I will look at the link. Still not sure about why everyone is talking about this XML stuff. But thank you for taking the time to respond to my post.

  74. #74 Miguel C. says:

    I think the problem is CPU overhead because real time update of Smart PLaylists. If you disable real time update in all of them it solves a lot.

    Miguel.

  75. #75 Bob Smith says:

    Re: #8 Jason…what a typically arrogant developer response - tell the user it’s his fault for having too large a music collection. You twit.

    “Maybe you simply have too much music.” Who are YOU to say anyone else has “too much music”? My own CD collection when converted at 128k was over 60 gig. Twit.

    “I think your complaints are unwarranted.” Typical developer response. The product has some architectural issues that’s causing a user problems, and you want to poo-poo the issues.

    “I don’t know anyone who has that much music.” Yea, well, I don’t know anyone who doesn’t own a car. Does that mean EVERYONE should own a car? Twit.

    “Maybe if you think it’s necessary to have that much music on your computer, you should consider Audion.” Yea, don’t look to the manufacturere for a solution to their mistakes-go find someone else to fix Apples’ flaws.

    “iTunes wasn’t intended to handle that much data.” A direct contradiction to “I think your complaints are unwarranted”. This is exactly the point he was making- iTunes development did not address large music collections, although this occurs with 60 G collections for the 60 G iPod.

    “They never thought that anyone would be sticking that many songs into the app.” Yea, right. That’s why this problem occurs with 60G file collections for the 60G iPod. They knew people would be doing this, just not the majority of users, so they were willing to accept this problem for the small group who would have problems.

    You’re a typical developer twit…shutup and go home.

  76. #76 NOISESHAPES says:

    I have a problem that I believe is also related with itunes database and that is that Itunes doesn’t see all the music from ipod. Other ipod file managers do see everything and also I don’t have a problem playing it from the ipod. This happened after I installed winamp ipod plugin ml_ipod or something like that which is known to conflict with itunes. I did re-install itunes (after uninstalling and cleaning refferences in registries in windows) and I did uninstall winamp and its plugin but still no change in itunes. Does someone know how I can solve this?
    thank you

  77. #77 SoCalSwami says:

    I AM NOT ALONE!
    ~
    For the longest time I have suffered in silence, searching the web for a magical cure-all that would end my agony. While this thread won’t resolve my pain, at least I know there are others (and rather articulate ones at that) who suffer they same fate as I.
    ~
    What the hell was Apple thinking when they designed the “itl”? What an awful excuse for a database.
    ~
    I tend to use iTunes as the transpoter to my iPod and listen on my iPod. For this I need an up-to-date library.
    ~
    Since I prefer iTunes not manage my music, any music I add manually to my library folder system directly is not updated in iTunes.
    ~
    The idea that iTunes does not have an automated library update features is jaw-dropping. Winamp does.
    ~
    To get around this problem I used two solutions.
    ~
    My first solution (up to about 10,000 songs) was to delete the itl and xml files and re-import the entire library. The would take roughly 2-6 hours import time with the gapless playback and volume leveling adding a huge chunk of process time.
    ~
    Then I found itlu (google: iTunes Library Updater) and life was good again. But, support seems to have stopped while the author is focused on school. And recent attempts to use itlu have not removed broken links and left out hundreds of new songs.
    ~
    Now at 42,000 songs I am simply screwed.
    ~
    It seems Apple is not only opposed to writing the code to update your library themselves, they clearly do not want anyone else writing such a tool.
    ~
    aND plEZ, enough talk about your iPod woes, your missing the point of the thread.
    ~
    And don’t get me started on the thought I have too much music. OMFG!

  78. #78 nodata says:

    maybe i have a fix ihad the same problems 35k songs on 400gig extdrive playback stoping every 10 to 35 secs banging my head inbetween so looks like you need to store only the song files on the extdrive keep itunes and xml files on the internal drive playing tracks and writing xmls at same time seems to much for firewire its working for me ta to all yo who posted xml was the key good key hint thank you thank you

  79. #79 Raymond says:

    I have close to 120k songs and iTunes is really getting slow here. Please Apple, do something with how you handle the database!

  80. #80 Randy W. says:

    Well, now I think Apple may have screwed themselves. They have not come out with a 160GB iPod and the only software that supports it cannot successfully handle that much music. I am just venting here, but, what are they thinking?
    I have just over 70,000 songs in my library and after burning all of my thousands of CDs (I am little older), I store my music on a server connected to this computer (both Windows variety) by a gigabit connection. My iTunes library sits at 50% utilization of the CPU (dual processor, so it is actually 100% of one core – which is all it can utilize) any time I do anything from trying to add a song, change songs or even scroll the list of songs.
    Now that Apple has come out with a 160GB iPod, how do they expect to handle the complaints of people who have these problems? I have moved my iTunes library from Windows 2000 to XP to Vista to my Mac and none of them can handle the bogging down of iTunes worthless database system. I suppose there is always hope for 8.0 because 7.4 lasted for just under a week and this morning upgrading to 7.4.1 I hoped for some satisfaction, but I was no less disappointed than going from 4 to 5, 5 to 6 or 6 to 7.
    I have 4 iPods and at this rate I am so frustrated that I think I may be ready to go with another mp3 player and just give up on Apple/Mac like I did back in the early 90’s.

  81. #81 Pilya says:

    To Randy W: I have to ask, what would you replace iTunes/iPod with? The fact of the matter is that there are no library systems for the Windows/Mac world designed to handle large libraries. My library is only 40,000 songs so I do not have the 70k experience. I can tell you that 40k in iTunes is a much better experience then 40k in winamp or wmp.

    The big move (if and when) will be in Leopard when they can move the “database” into a real database ala Core Data. (note: iTunes is not in a database at all right now… its strictly a xml file… huge difference).

    (if and when) Core Data is used, I am 1000% confident that all of our performance woes will be dramatically changed. But then again… you are using windows… so in keeping full windows computability there is a monstrous problem in that the cocoa framework does not live in windows at all. So as they did with safari, they will need to port more cocoa framework goodness… not an easy task.

    In the meantime if you do find a system that is as joyous as the iPod world… please let us know. I dont know of a single entity that provides an equal or greater Ux then what you currently have.

  82. #82 S McGee says:

    I am so disappointed with my latest version of iTunes because the quality in which it plays the movies is so poor that it is no longer worth it to purchase downloads and new DVD’s. In fact Apple has gone from a company I have great respect for to one that is just like all the rest. What disappointment. My “entertainment” center has turned into another job that I have to keep on updating and fixing.

  83. #83 DvdMovies says:

    All I had to do was uninstall and reinstall Quick Time Player. I use Windows and not too computer savy - maybe this could help someonelse that is also computer illiterate.
    lucky You, I can’t do that…

  84. #84 W R D says:

    Could someone figure this out for me….I only have 500 songs, but everysong has this popping like issue happening, you know, like their was dust on the record player needle. When I purchased the PC itunes was already installed, but I did one of the updates and everysong went for crap. Some of the songs I cannot get again and every song was done from a new cd. Can anyone tell me what happened???

  85. #85 Raymond says:

    Apple’s support forums would be the place for you to ask.