One Lightroom Catalog or Many?

[Marianne]

Do you have one huge Lightroom catalog with all of your images or do you have several?  How do you divide them up if you have more than one?

 

I thought this would be an interesting discussion and I also could use some advice. 

 

I have one huge catalog. Organization is tricky for me, thanks to ADD, so I like having everything in one place, but with over 100K images, it is just so large I'm not sure that one catalog makes sense anymore. 

 

Back in 2011, I organized all of my photographs going back to 2005 from various backup drives and CDs/DVDs (remember them?), into one giant LR catalog. It took me over a year to get through a slew of disorganized drives and discs and determined which was the most current version. But the hard work paid off and I was really happy with how it turned out.

 

But now it is getting unmanageable. The current catalog is backed up to two duplicate  drives - a 4TB LaCie and a 6TB G-Tech - and most of it is also on a WD RAID array that got too small for the entire thing, so I split it into one for Europe and one for the US, leaving only my US photos on the RAID and backing up the Europe catalog (much smaller) onto another drive. I also have several smaller backup drives - mostly 1-2TB - with smaller "working" catalogs of various sizes - which I find to be far less organized than the two large ones. Since I often rework images or have shoots where I've only developed a handful of images (I used to use the Nik Capture software which was so slow compared to LR where I can process a slew of images at once), I find that having everything in one place is helpful, but the catalogs have gotten so huge now that I'm considering going back to a smaller catalog system, but could use some organizing tips. 

 

I'm considering using the two huge catalogs primarily as archives - and starting a new large "working catalog" for each year, then backing them up as I go to the large archive, so that I'd work on older images from the backup drives, while keeping everything current - the last two years - on my laptop (with 2 backups). 

 

I also back up everything to Sync and Photoshelter on the cloud, but neither of these back up the catalogs - just the photos - which means extensive edits in Lightroom, stars, colors, organization, are only on the hard drives.  Is there any online backup solution for catalogs? 

 

Please discuss. Thanks!

[Sally]

You might want to pose this question on the Lightroom Queen forum https://www.lightroomqueen.com/community/

You can, of course, back up your catalog ie the xmp files etc. I back mine up automatically on my hard drive, and also on an external drive, and it all gets backed up off site via backblaze. So far I have only one catalog of photos, but mine won’t be as big as yours. The photos are on an external hard drive, which is backed up to another external hard drive, as well as occasionally onto a portable drive, and all backed up offsite via backblaze. After one of my hard drives failed recently, it spurred me to get everything sorted properly.

[MDM]

Maybe I'm wrong but I wonder if you are perhaps a little confused as to what a catalog actually is and the difference between a catalog and an archive of files. I'm not sure what the real problem is or whether there is one as you are just saying it is too big but not saying why that is problematic. If Lightroom is running slowly then that might be a reason for starting a new catalog but at around 100K images I don't think that should be a problem in itself.

 

A catalog is just an advanced database really and doesn't contain any files, just information about the files. As long as you can access the information easily, the size of the catalog should not be a problem and should not really be a reason for splitting it up, especially if you like having everything in one place as you say. You can keep files on multiple disks which do not have to always be connected to the catalog disk. I have one big catalog of 79,000 files of my landscape and general stock work, The files are on two separate master Thunderbolt drives as well as being backed up on other drives and the catalog is on an internal SSD. I don't have an automatic online backup.  

 

My tips for organisation are:  have good metadata associated with the images so they can be found rapidly and Lightroom has a very good and fast search engine for that purpose. I always advocate keeping as much of the info independent of the catalog so I set "automatically write to xmp" as always on so it writes the metadata for raw files to xmp sidecar files (other files have the metadata embedded). If the catalog corrupted then I still have all the metadata. In addition I don't think it is a good idea to rely on Lightroom or Adobe  specific feature such as labels and ratings. I think they are very handy when trawlling through pics from a shoot but I always use keywords in addition as these travel with the files and could be read by other software.

 

I do have other catalogs for various purposes (clients, tests ....) but that is another story related to the purpose of the images really. The main point really is that catalog size should not be in itself be a problem as long as you have reasonable metadata.

[Marianne]

Thanks for your replies.

 

I do automatically write everything to xmp for my RAW files and I keyword my images automatically as I upload them to my computer/backup drives via Lightroom, so I can find them independent of the catalog.

 

I have my catalog on my iMac hard drive with backups on 2 external drives. I guess because I have organized all of the files associated with that catalog on the same 2 backup drives (they are clones of each other), I felt that I was running out of room. If I split the photos associated with the catalog onto different external drives, and keep the catalog intact, then that will work. Thanks for helping me to realize where the problem lies.

As long as I sync the folders that I'm working on before signing out and backing up my catalog, or moving images (which I do via Lightroom so it keeps track of their location), even if something happened to the catalog, I'd have all my changes other than any Virtual Copies, and with catalog backups that should be okay. I think splitting the photo locations will do the trick. I use two computers, my laptop and my iMac - so I've always had to be careful about keeping them synced.  I just wish that I could keep my catalog on a network drive (prohibited by Adobe) or back it up offline as well as onto my external and portable drives.

 

Again, thanks for sharing your thoughts. 

 

 

 

[MDM]

21 hours ago, Marianne said:

Thanks for your replies.

 

I do automatically write everything to xmp for my RAW files and I keyword my images automatically as I upload them to my computer/backup drives via Lightroom, so I can find them independent of the catalog.

 

I have my catalog on my iMac hard drive with backups on 2 external drives. I guess because I have organized all of the files associated with that catalog on the same 2 backup drives (they are clones of each other), I felt that I was running out of room. If I split the photos associated with the catalog onto different external drives, and keep the catalog intact, then that will work. Thanks for helping me to realize where the problem lies.

As long as I sync the folders that I'm working on before signing out and backing up my catalog, or moving images (which I do via Lightroom so it keeps track of their location), even if something happened to the catalog, I'd have all my changes other than any Virtual Copies, and with catalog backups that should be okay. I think splitting the photo locations will do the trick. I use two computers, my laptop and my iMac - so I've always had to be careful about keeping them synced.  I just wish that I could keep my catalog on a network drive (prohibited by Adobe) or back it up offline as well as onto my external and portable drives.

 

Again, thanks for sharing your thoughts. 

 

 

 

 

Glad it helped. Moving files in Lightroom is a good way to do it but not essential. If it can't find a file or folder, then a question mark will appear and you can just point LR to the new location. You can back up the catalog an external drive by simple drag and drop or by setting it to backup to a non-default location in the catalog prefs. I use the drag and drop method frequently.

[Marianne]

4 hours ago, MDM said:

 

Glad it helped. Moving files in Lightroom is a good way to do it but not essential. If it can't find a file or folder, then a question mark will appear and you can just point LR to the new location. You can back up the catalog an external drive by simple drag and drop or by setting it to backup to a non-default location in the catalog prefs. I use the drag and drop method frequently.

 

Thanks. I know that I can always find lost files and even lost folders. I've been using Lightroom since v. 1.0 - but I was starting to question the fact that I'd made this monster catalog and didn't think about the fact that I can point the catalog to more than one backup disk, so that the catalog size isn't really important. I find that using Lightroom to move my files saves me that extra step - I really like it for organizing as well as editing - and for searching for photos since it is visual so I can see the photo(s) once it's/they are found - by number or keyword - and quickly be sure that I've found what I'm looking for. Sometimes an editor or client will be looking for a location and it's an easy way for me to quickly review what I have and send them previews - or even here with the monthly contests if I don't have something on Alamy already I can search and find something new (or old) to upload. 

 

I'm getting a new Mac so I'll need a hub to connect my various hard drives and it will be easy to split my collection and still keep it all in one catalog.

 

Again, thanks all for the advice. Sometimes the answer is so obvious but you miss it anyway. Glad you all have got my back. 

[Steve F]

Hi Marianne,

First off, I should admit that I'm not very au fait with organising photos or Lightroom. But, I have about 12 separate LR catalogues, organised by genre or geographically. I'm probably doing something wrong though, my photos only number up to 20,000 so I start getting some repeats (buying cameras too often!) and I need to create new LR folders. Do you rename your photos? Or probably some easy fix in LR I'm not doing
 

[Sally]

38 minutes ago, Steve F said:

Hi Marianne,

First off, I should admit that I'm not very au fait with organising photos or Lightroom. But, I have about 12 separate LR catalogues, organised by genre or geographically. I'm probably doing something wrong though, my photos only number up to 20,000 so I start getting some repeats (buying cameras too often!) and I need to create new LR folders. Do you rename your photos? Or probably some easy fix in LR I'm not doing
 

You should set LR to automatically rename your images on import so you dont get repeats. It’s easy to do in The import panel - just choose custom filename and set it to whatever you want. Usually good to include the date format, eg 17012019[filename] then you won’t get repeats.

 

Have as many LR folders of images as you want but only one catalog.

[Steve F]

Thanks LawrensonPhoto and Sally, definitely something I will start doing from now on.......! Been to busy editing and uploading to check out how to curate properly, my bad!!

 

I think whether or not to have multiple catalogues has been answered with a resounding no! So, to summarise, one LR catalogue, then save the photos in separate folders as you like.

[Starsphinx]

21 minutes ago, LawrensonPhoto said:

Yep, specially with several cameras on the go

I import thus: [location]_[date]_[number].jpg

Never had a repeat

Or better if you use photo mechanic to ingest and cull your images

Photo mechanic all the way for me - I ingest (when cant they use the word import like everyone else lol) using seq - date - subject in folders named date - subject.  Easy to arrange in order, easy to spot which folder I want.  When I need photos from multiple folders I use Lightroom collections.

[Marianne]

6 hours ago, Steve F said:

Hi Marianne,

First off, I should admit that I'm not very au fait with organising photos or Lightroom. But, I have about 12 separate LR catalogues, organised by genre or geographically. I'm probably doing something wrong though, my photos only number up to 20,000 so I start getting some repeats (buying cameras too often!) and I need to create new LR folders. Do you rename your photos? Or probably some easy fix in LR I'm not doing
 

I see you’ve gotten answers that yes you should rename. I’ll just share the system that has worked well for me.  

 

I’m in the US so I use the state abbreviation for travel & nature pix (or country) and then go with 4 consecutive numbers so I’ll have NY18-0123, CT18-1234 and for still life I’ll use a descriptive term such as Pumpkin18-5666. I keep the numbers consecutive starting with 0001 for the year so with multiple cameras I have a sense of how many photos I’ve shot. I prefer a descriptive name to the date, but that’s purely personal preference. 

 

I also like the 4-didgit naming because some clients require me to use my name or initials to rename files so I can have  a duplicate named MC18-1234 and if I want to find the original of that  photo later I’ll know it’s NY18-1234 or Pumpkin18-1234. I also use the same 4 digits in the names of any different fine art versions of an image  so I can again trace back how many different versions I’ve done. I might end up with NY18-1234-texture, NY18-1234-lowers, NY-1234-BW etc.

 

 I’m unlikely to shoot more than 9999 photos in any one year so it works for me. I’ve used a substantially similar naming convention since the mid-2000s.

 

Pick something that works for you and rename and do basic keywording as you upload your photos via Lightroom and when you hit 100,000 photos, you’ll be glad you did. 

[Bryan]

I'm a peripheral dabbler with LR, so can't offer any sound advice, but I do have a problem with response times when entering keywords.

 

Occasionally when I start to enter a commonly used string of characters the software briefly hangs up - just a second or two - and I was wondering if I was storing too much information, in particular too many keywords.  For the record I have around 22,000 images and a single catalogue. I do virtually all of my keywording in LR, so there is lots of text. Would using a second catalogue alleviate this problem, or would it mean that I could not access the keywords between catalogues?

[MDM]

1 hour ago, Bryan said:

I'm a peripheral dabbler with LR, so can't offer any sound advice, but I do have a problem with response times when entering keywords.

 

Occasionally when I start to enter a commonly used string of characters the software briefly hangs up - just a second or two - and I was wondering if I was storing too much information, in particular too many keywords.  For the record I have around 22,000 images and a single catalogue. I do virtually all of my keywording in LR, so there is lots of text. Would using a second catalogue alleviate this problem, or would it mean that I could not access the keywords between catalogues?

 

I've never seen Lightroom have a problem with keywords and 22,000 images is not a very large catalog. It is probably something to do with autofill and likely to be normal behaviour. See the Catalog Prefs - Metadata tab - Offer Suggestions from Recently .... You could try unticking and see if it stll happens. Also always a good idea to occasionally optimise the catalog.

[Marianne]

3 hours ago, Bryan said:

I'm a peripheral dabbler with LR, so can't offer any sound advice, but I do have a problem with response times when entering keywords.

 

Occasionally when I start to enter a commonly used string of characters the software briefly hangs up - just a second or two - and I was wondering if I was storing too much information, in particular too many keywords.  For the record I have around 22,000 images and a single catalogue. I do virtually all of my keywording in LR, so there is lots of text. Would using a second catalogue alleviate this problem, or would it mean that I could not access the keywords between catalogues?

 

You might have some near duplicates stored, which is the only thing I can think of that might possibly slow you down, but I have over 80,000 photos in my main catalog and I don't have any lags with keywording using either my current late 2013 Macbook Pro with a 2.8 GHz Intel Core i7 and 8 GB RAM nor with my older iMac, so I don't think the size of your catalog or your equipment is likely to be the problem. 

 

Do you automatically optimize your catalog? I always back up my catalog automatically when I close it after I'm finished working at the end of the day, and I have "optimize catalog" ticked. It takes a few minutes for the catalog to back up and optimize that way. I have only had one instance since starting with Lightroom v.1.0 several years ago where I had to restore my catalog. I go through the backups every month or so and cull them when they start to take up too much space, keeping a few in case I ever have a problem. I'm not sure how long it would take to optimize the catalog if you've never done it, but once you do I'd keep it ticked. 

 

I do all my keywording in Lightroom - I enter basic keywords when I first import images from a shoot, usually from metadata presets that I have for places and subjects I tend to shoot often, and then add or delete new keywords for specific images after I have reviewed and culled the imports, which I may then refine further during processing and/or before uploading. 

 

When you quit the catalog, you have the option of clicking on "optimize catalog." I'd try that and clearing out any misspelled keywords (before deleting them you can search your catalog for any photos with that keyword and correct it) since having several that are too similar might possibly be what is slowing things down as it autofills them for you. 

 

I think you can save metadata presets in your Adobe Account and then use them between catalogs, but I'm not certain. Since I currently use two computers, I have a smaller catalog on my laptop so I can cull and keyword images without the need for attaching another drive and then I export the finished folder as a catalog to an external drive and import it into my main catalog, deleting the exported catalog when I'm done and just leaving the images. It's not the most efficient way of doing things, but it works well enough for me. I'm hoping that once I upgrade my Macbook to a 2TB drive (I wanted the 4TB but it would cost just under $7,000, which is insane for a laptop), I can keep a copy of my catalog on the laptop, with newer photos, and leave the older ones on my backup drives.

 

Where I've gotten into trouble with multiple catalogs in the past is when I've made changes to some images on my iMac and then edited the same shoot on my Macbook so that I end up with two versions of the same shoot. Once I put together one huge catalog on an external drive, I knew that any images that were only on my Macbook would be backed up to a different portable external drive, so I would not duplicate them by adding them to the main catalog until they were culled and keyworded. The ability to sync folders in Lightroom now makes the move between computers easier, but it still wastes a lot of time and I'm really looking forward to having one computer and one monitor so I don't need that extra step. 

 

[Bryan]

Thanks for the optimise suggestion, new one on me, but autofill also sounds a likely culprit - however too useful to turn off!

 

Actually, thinking about it,  the software asks me about once a week if I want to do something to the catalogue, never really paid any attention, but I guess it wants to optimise. I generally allow it to do so.

[DHill]

On 19/01/2019 at 05:36, Bryan said:

I'm a peripheral dabbler with LR, so can't offer any sound advice, but I do have a problem with response times when entering keywords.

 

Occasionally when I start to enter a commonly used string of characters the software briefly hangs up - just a second or two - and I was wondering if I was storing too much information, in particular too many keywords.  For the record I have around 22,000 images and a single catalogue. I do virtually all of my keywording in LR, so there is lots of text. Would using a second catalogue alleviate this problem, or would it mean that I could not access the keywords between catalogues?

Switching off auto-complete will probably help, but there may be another thing happening, too. I find that if I inadvertently pause slightly between, say, the second and third characters of the word I'm entering, LR immediately hangs while it searches for all the kws that include the string I've already typed. It's just an annoying quirk of LR, I'm afraid - something they haven't rectified in a number of versions.

 

Having said that, I'm gaining a renewed appreciation of LR's handling of key words: I'm switching to Capture One for image processing, which it does far better than LR, but the boot's on the other foot for kws - LR deals with them much better than Capture One.

[RedSnapper]

I just have one LR catalogue.....which, as of today, contains the data on 933,884 images and is 10.23gb

 

km