JeffGreenberg

how to retrieve thousands of photos at once from dozens of folders...?

Recommended Posts

(using Windows 10)(images stored in external HD)

need to retrieve copies of 12,315 TIFs from dozens of

different folders & put them all in one new folder;

(about 10% of total TIFs in all those folders)

Let's call them Image1, Photo2, File3...Picture12315.

Trying the following in Windows search box for just (3) returned nothing:

Image1, Photo2, File3

Image1 Photo2 File3

Image1; Photo2; File3

 

How can I accomplish this???!! 

Thanks in advance.

Otherwise its one image at a time... :( :( :( 

Edited by JeffGreenberg

Share this post


Link to post
Share on other sites

I know you don't want to learn Lightroom, Jeff, but it has many ways of searching for images -- camera used - text - keywords, etc. I guess it would be a big job to get your images into the program but probably would be worth it in the end.

 

Paulette

  • Like 1

Share this post


Link to post
Share on other sites

Any chance you added tags or keywords in metadata?  Or you have certain date ranges in mind?   Or a particular camera they were taken with?  You have my sympathies - I am horrendously bad at organizing and have got into stupid muddles myself - which have only resulted in attempts at better systems than going 2 years just titling every folder with the date (and I am generally creating a folder every couple of days - some of which will run to 4 figures of photos - quite possibly mixed)

Share this post


Link to post
Share on other sites

My images are organized geographically, but the 12315 I want to retrieve

are dozens of destinations, dozens of years, multiple cameras.

I have no problem finding one at a time, I have Excel doc of all 12315,

I am hoping to call them all up at once...but how?

How does inserting 12315 file names into Lightroom call up all

12315 images out of 132K total images?  I have LR & can do that if told how.

Edited by JeffGreenberg

Share this post


Link to post
Share on other sites
40 minutes ago, JeffGreenberg said:

My images are organized geographically, but the 12315 I want to retrieve

are dozens of destinations, dozens of years, multiple cameras.

I have no problem finding one at a time, I have Excel doc of all 12315,

I am hoping to call them all up at once...but how?

How does inserting 12315 file names into Lightroom call up all

12315 images out of 132K total images?  I have LR & can do that if told how.

If I have understood you correctly, this should be possible. Import your photos into LR. This doesn’t move them, it just creates a catalog of your images. You can import them from any hard drive. Then you have access to them all in one place. You don’t say what defines the 12,315 images you seek. What links them? 

Share this post


Link to post
Share on other sites
7 minutes ago, Sally said:

You don’t say what defines the 12,315 images you seek. What links them? 

 

They all need to be submitted to an agency.

There is nothing in their metadata or text fields that links them.

They were not chosen, they were "derived" by subtracting one

larger group of image names from a smaller group of image

names to yield a remaining group of 12315 image names.

Now I have to gather 12315 actual files...

Share this post


Link to post
Share on other sites

This is a classic case of where proper DAM (Digital Image Management) software is needed.  This could be done easily with IMatch, using the CSV import module .....

 

https://www.photools.com/help/imatch/#impexp_csv.htm

 

Create another column in the Excel file and copy something into every row (eg. the name of the agency).

Export the Excel file as a CSV file (filename must be the first column).

Open the Imatch import module - fill in the details and select which metadata tag you want to enter the data. You would only need to import the filename and that extra column (data goes into the sidecar XMP file, not the image file).

Once the importer has done its thing you then tell IMatch to display all files with that tag - all will be displayed regardless of which drive they are on.

Select all files, select 'copy' (or move), select 'paste' into the new folder.  Job done.

 

Of course all the files have to be imported into the IMatch database before it can find them.

 

Also of course it doesn't really help in your present situation but I give it as an example of what can be done with true DAM software.

 

 

Edited by Vincent Lowe
  • Upvote 1

Share this post


Link to post
Share on other sites

One more vote for IMatch

 

I don't think Lightroom was intended to be a DAM solution...?

  • Upvote 1

Share this post


Link to post
Share on other sites
15 minutes ago, LawrensonPhoto said:

Easy, adobe bridge and free ;)

 

I am just wondering how this could be done in Bridge given that there is no metadata or anything else identifying the images in question apart from the file names. Maybe I am missing something but I can't see how this could be done. Also I am guessing that Jeff probably has an image collection of let's say 200,000 images (probably very conservative). In my pre-Lightroom days when I used Bridge, I found it was terrible with even moderately large image numbers (say 5000 or so), losing the cache and requiring the whole cache to be rebuilt every so often. Dare to move a large folder and well enough said (in my opinion of course).

 

35 minutes ago, dlmphotog said:

One more vote for IMatch

 

I don't think Lightroom was intended to be a DAM solution...?

 

Lightroom was definitely intended as a DAM and it does a damn good job as a very fast database if the metadata is present. Searches are very fast even with very large catalogs. However Lightroom is not customisable in the way that Vincent describes for IMatch and it can't do the job in question here as there is no metadata in the files. I find that Lightroom does most of what I need as a DAM and the integrated raw converter is not too bad at all.

Share this post


Link to post
Share on other sites

Is Windows search box really limited to returning only one item at a time...?

Are there no settings to make it multi-item searchable?

Or by placing some symbol between two items? % # * or?

Is there any search box that can return two files at a time from different folders?

Share this post


Link to post
Share on other sites

Adobe Bridge has Edit>Find

& then allows more than one row of search boxes

but only allows one file name per row AFAICT...

one still has to paste file names one by one...

Share this post


Link to post
Share on other sites

There is a new version of DxO's PhotoLab just out (PhotoLab 2) that has an improved search function. You can enter more than one search term per search (location, camera, ISO, etc). You don't have to import anything, just have the HDs available. Free trial enough to let you know if it is going to work or not. 

 

Disclaimer: I'm not well organized and don't get on with software too good. Upgraded yesterday, tried the search today and found that it does work. Not sure it works perfectly, but there is a high degree of operator error involved. HTH

Edited by KevinS

Share this post


Link to post
Share on other sites

In bridge you can see the contents of folders and copy the contents, doesn’t matter how many folders and copy the images to another folder which is what he wants no? 

Share this post


Link to post
Share on other sites

Have you considered using the xcopy DOS command, it has some powerful options (and can be run in a DOS batch file to automate the entire process).

 

Open a command window by typing cmd in the Windows 10 search box.

Navigate to your external HD by typing the drive letter followed by a colon and then hit carriage return (enter)

For example

Z:<CR>

 

Now navigate to the folder that contains all the subfolders with your images in using the cd (change directory) command

For example

cd Z:\my pictures\target folder<CR>

 

You can see the files in the folder you've selected by typing dir  if you need.

Now type xcopy/? to see all the options the xcopy command provides. Of particular note are the following options.

 

/S which allows files in subfolders of the current folder to be accessed

/P which will offer a yes/no prompt before copying each file

/EXCLUDE: which allows a list of files to be excluded from the copy (shame there's not an INCLDE: option)

Hitting <Up arrow> displays previous command and allows it to be edited.

Wildcard file names are supported (e.g. *.tif)

/? shows all options for any dos command

 

So the following commands might help achieve what you want

Firstly make a new directory to receive your files using the mkdir (make directory) command, for example

mkdir newfolder<CR>

 

The do a prompted copy of all tifs from any subfolders

xcopy  *.tif newfolder /S /P

 

This will show the name of each file before you choose (manually) whether to copy it or not. Far from ideal.

 

However, now you have the principle. If you have a list of the filenames you want to copy you could create a DOS batch file to run a sequence of dos commands to do the copying one file at a time. If you have a list of the filenames in Excel it should be easy to use the concatenate option to add the the guts of the xcopy command around each filename. Then save as a batch file and the just run it as batch job.

 

Let me know if you need help. I can be contacted via the contact button here.

https://fineartamerica.com/profiles/photimageon-UK.html

 

Mark

Edited by M.Chapman
  • Upvote 1

Share this post


Link to post
Share on other sites

So if I have d: drive and under d: I have 100 folders and within that I have a thousand folders and within all those folders I have 20000 images I can copy and paste all of them to another folder

Share this post


Link to post
Share on other sites
11 minutes ago, LawrensonPhoto said:

In bridge you can see the contents of folders and copy the contents, doesn’t matter how many folders and copy the images to another folder which is what he wants no? 

 

No he wants to retrieve his 12,315 images automatically by a search based on the file name alone which he has as a list in an Excel file from a much larger set, not pick them one at a time or even several at a time as Mark seems to suggest. I think Vincent Lowe's solution is probably the only one that would work sensibly without several days manual labour but I am not familiar with Imatch. 

Share this post


Link to post
Share on other sites

Actually, I've just been re-reading that IMatch help file I linked to and realised it's not as simple as I first thought. The first column of the CSV file has to be a 'fully qualified' file name - in other words it has to contain the full path to the folder, such as D:\images\florida\beach.jpg

It won't work with just the file name.

 

Share this post


Link to post
Share on other sites
14 hours ago, JeffGreenberg said:

My images are organized geographically, but the 12315 I want to retrieve

are dozens of destinations, dozens of years, multiple cameras.

I have no problem finding one at a time, I have Excel doc of all 12315,

I am hoping to call them all up at once...but how?

How does inserting 12315 file names into Lightroom call up all

12315 images out of 132K total images?  I have LR & can do that if told how.

 

This is possible with Lightroom I believe using this plugin: https://www.photographers-toolbox.com/products/lightroomstatistics/photolistimporter/

 

Import all images from into Lightroom then using this plugin send your list of filenames to Lightroom to obtain the required subset of images which the plugin puts into a new collection. Finally export the images form this collection from Lightroom in whatever image format required ready to submit. Apart from the time taken for Lightroom to ingest all images and then export them, this should be relatively straightforward. I've never used this plugin so can't vouch for it however. Probably worth testing on a small number of images in lightroom first.

 

To speed up the importing, set build previews to 'minimal' in the import panel as you don't need Lightroom to create large image previews for this task as these are only for viewing images within Lightroom. All of these taks will take a long time so it might be worth breaking up each one into smaller bits rather than do it all in one go.

 

Craig

Share this post


Link to post
Share on other sites
1 hour ago, MDM said:

 

No he wants to retrieve his 12,315 images automatically by a search based on the file name alone which he has as a list in an Excel file from a much larger set, not pick them one at a time or even several at a time as Mark seems to suggest. I think Vincent Lowe's solution is probably the only one that would work sensibly without several days manual labour but I am not familiar with Imatch. 

 

Although my proposal copies files one at a time, with a batch file created from Jeff's Excel list I could automate the entire process. Should only take an hour or so to put together a suitable batch file if Jeff gives me the details needed. However I'm not sure if there's a limit on the size of a batch file which could throw a spanner in the works.

 

Update: I just checked and there should be no limit on batch file size although there maybe a 127 character limit on each line, so I hope Jeff's filenames aren't too long.

 

Mark

Edited by M.Chapman
  • Upvote 1

Share this post


Link to post
Share on other sites
Quote

I have Excel doc of all 12315,

Jeff, Do you have the full pathname of these files (directory/subdirectory/filename)?

 

If so I would create a simple command file along the lines copy sourcefile destination folder (easy to do in Excel) then run it as a command in a "Dos" (remember that?) window.

Share this post


Link to post
Share on other sites
1 hour ago, Russell said:

Jeff, Do you have the full pathname of these files (directory/subdirectory/filename)?

 

If so I would create a simple command file along the lines copy sourcefile destination folder (easy to do in Excel) then run it as a command in a "Dos" (remember that?) window.

 

Jeff's got two offers now. Isn't the forum wonderful. :-)

If Jeff doesn't have the full path (only the filenames) I think XCOPY with /S option could work?

 

Mark

  • Upvote 1

Share this post


Link to post
Share on other sites

Thanks for all responses.

DOS advice passed on to wifey.

Paths to specific folders diverge almost immediately

a. FL or non-FL

b. in FL: SouthFL or non-SouthFL

c. in SouthFL: years 2003 to 2018

d. in non-SouthFL: years & city names

e. in non-FL: years & country names

 

all this means hundred+ different paths;

my 8TB drives might have enough unused space

to copy ALL 132K 8bitTIFs into single folder,

(even that will take time to do)

then auto-pull 12315 of them into another empty folder...?

 

 

Still hard to believe Windows 10 search box can't search

more than one file at the (FL or non-FL) level...

Maybe finding a Windows 10 forum is prudent...?

 

Share this post


Link to post
Share on other sites
2 hours ago, JeffGreenberg said:

Thanks for all responses.

DOS advice passed on to wifey.

Paths to specific folders diverge almost immediately

a. FL or non-FL

b. in FL: SouthFL or non-SouthFL

c. in SouthFL: years 2003 to 2018

d. in non-SouthFL: years & city names

e. in non-FL: years & country names

 

all this means hundred+ different paths;

my 8TB drives might have enough unused space

to copy ALL 132K 8bitTIFs into single folder,

(even that will take time to do)

then auto-pull 12315 of them into another empty folder...?

 

 

Still hard to believe Windows 10 search box can't search

more than one file at the (FL or non-FL) level...

Maybe finding a Windows 10 forum is prudent...?

 

 

Hi Jeff,

 

It doesn't matter that there are 100s of paths, so long as they all start from the same root folder. I just tried XCOPY on my system and, using wildcard filenames it found any files that met my wildcard filename criteria from wherever they were buried on my system and copied them to a new folder. It should be noted however that it did create the same directory structure within the new folder to hold the files it copied. But it's easy to combine them into a single folder in Windows using a regular search for *.tif, the select all and cut and paste into a new single folder (assuming all your filenames are unique). 

 

I'm pretty sure that Russel or I can get this working if you want (subject to the caveat in bold above if you don't have the full path to each file). If you can email me the Excel file containing the filenames you want to find then I'll construct a batch file to automate the process. If you want to contact me to exchange email addresses then use the contact button here https://fineartamerica.com/profiles/photimageon-UK.html

 

Even if Windows 10 can search for several files at once, I very much doubt it can search for 100s of filenames at once, let alone 1000s

 

Mark

 

 

Edited by M.Chapman
  • Upvote 1

Share this post


Link to post
Share on other sites

MarkC & all,

 

Thanks for generous offer, but...

 

Doah.  Why didn't I search Google before now??!!  Got this & it works:

(the separator that works is "OR" & must be capitalized)

https://www.addictivetips.com/windows-tips/search-multiple-file-types-in-file-explorer/

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now