Jeffrey Isaac Greenberg Posted May 1 Share Posted May 1 PS AI Denoise: why not all images? save original DNG? have now used it on batch of (27) ISO 2000--6400 & seen impressive results; any drawback using down to ISO 1000 or even ISO 100? (other than unnecessary?) noticed it turns (RX10 IV) 30MB DNGs into 50MB DNGs; is all data still contained in latter? Any reason to also save original DNG? what is easiest way to batch change AI Denoise file names back to what they were, e.g., MX231221001-enhanced-NR.dng FL231223100-enhanced-NR.dng etc., back to MX231221001.DNG FL231223100.DNG etc. thanks in advance, regards jg Link to comment Share on other sites More sharing options...
Betty LaRue Posted May 1 Share Posted May 1 3 hours ago, Jeffrey Isaac Greenberg said: PS AI Denoise: why not all images? save original DNG? have now used it on batch of (27) ISO 2000--6400 & seen impressive results; any drawback using down to ISO 1000 or even ISO 100? (other than unnecessary?) noticed it turns (RX10 IV) 30MB DNGs into 50MB DNGs; is all data still contained in latter? Any reason to also save original DNG? what is easiest way to batch change AI Denoise file names back to what they were, e.g., MX231221001-enhanced-NR.dng FL231223100-enhanced-NR.dng etc., back to MX231221001.DNG FL231223100.DNG etc. thanks in advance, regards jg This doesn’t answer your question, but I am deleting all the DNG-enhanced files once I save the jpeg for uploading. That leaves me with my original RAW with the jpeg next to it. Why delete it? I never revisit the image once I develop & submit it. Do you? Saving the DNG just eats up space, & if, I say IF, I should ever want to redevelop one, that’s a good trade off as far as I’m concerned, to start from scratch on a one-off exception. Link to comment Share on other sites More sharing options...
Steve F Posted May 1 Share Posted May 1 Unnecessary for low ISOs. DNG takes up file space. Agree with Betty. Link to comment Share on other sites More sharing options...
M.Chapman Posted May 1 Share Posted May 1 (edited) See I wouldn't keep DXO DNGs because they are so big. I just keep the original RAWs the sidecar files and finished jpgs When I was using Windows I found this free batch/bulk file renaming utility did everything I needed. It's extremely powerful. https://www.bulkrenameutility.co.uk/ Mark Edited May 1 by M.Chapman Link to comment Share on other sites More sharing options...
MDM Posted May 1 Share Posted May 1 6 hours ago, Jeffrey Isaac Greenberg said:contained in latter? Any reason to also save original DNG? what is easiest way to batch change AI Denoise file names back to what they were, e.g., MX231221001-enhanced-NR.dng FL231223100-enhanced-NR.dng etc., back to MX231221001.DNG FL231223100.DNG etc. thanks in advance, regards jg You can rename files very easily in Lightroom or in Bridge back to the original by doing a custom rename and choosing Preserved Filename. No need for additional software. However, if the originals stlll exist and are actuall DNG files, there will probably be a naming conflict. 1 Link to comment Share on other sites More sharing options...
MDM Posted May 1 Share Posted May 1 (edited) If you have done any editing on the enhanced DNG file and don't want to keep it to save space but do want to retain the edits, a simple solution is to synchronise the DNG with the original raw file so the raw will have all the edits preserved in the sidecar xmp file. Then if you ever want to make changes, you edit the raw and run it back through Denoise. Edited May 1 by MDM Link to comment Share on other sites More sharing options...
Jeffrey Isaac Greenberg Posted May 2 Author Share Posted May 2 thanks for responses; space not an issue; currently have (3) 8TB Seagate external HDs as "identical" backups; 1.61TB free of 7.27TB & doubtful they'll ever fill; if they did, then upgrade to (3) 10TBs... I keep DNG, 8bitTIF27MB, & JPG27MB for each image; (TIF processing via 16bit full size, but not saved, too big at ~110MB ea.) am going to assume DNG-enhanced-NR can be converted back to original DNG if ever required, so not saving (2) DNGs going forward... but no one confirmed it here yet... I also have no idea why one would want DNG-NR ==> DNG, but... Link to comment Share on other sites More sharing options...
MDM Posted May 2 Share Posted May 2 (edited) 1 hour ago, Jeffrey Isaac Greenberg said: thanks for responses; space not an issue; currently have (3) 8TB Seagate external HDs as "identical" backups; 1.61TB free of 7.27TB & doubtful they'll ever fill; if they did, then upgrade to (3) 10TBs... I keep DNG, 8bitTIF27MB, & JPG27MB for each image; (TIF processing via 16bit full size, but not saved, too big at ~110MB ea.) am going to assume DNG-enhanced-NR can be converted back to original DNG if ever required, so not saving (2) DNGs going forward... but no one confirmed it here yet... I also have no idea why one would want DNG-NR ==> DNG, but... It's not possible to unDenoise a DNG. How could this be done? As I said above, if you have edited the enhanced DNG, you can keep the settings by synchronising with the original in ACR or Lightroom. Most of use don't have original DNGs. My originals are mostly Nikon NEFs. On another note, your disks are already close to full as it is considered very bad practice to fill disks completely. The usual recommendation is don't go beyond 75-80% full or therabouts. You can read about this from a web search. Not what am I doing lying awake in the middle of the night answering questions on the Alamy forum? I must be nutz. Edited May 2 by MDM Link to comment Share on other sites More sharing options...
M.Chapman Posted May 2 Share Posted May 2 (edited) A quick look at file sizes shows some big differences (RAW / Adobe Denoise DNG / DXO PureRAW DNG) 20MP Sony RX100 Mk3 - (21MB / 49MB / 85MB) 20MP Lumix G100 (24MB / 39MB / 75 MB) 46MP Nikon Z7 (91MB / 109MB / 199 MB) It appears DXO PureRAW DNGs are typically about double the size of Adobe Denoise DNGs Not sure why the Nikon Z7 RAWs I downloaded from DPReview are so large. Others I've seen are around 50-60MB. Mark Edited May 2 by M.Chapman Link to comment Share on other sites More sharing options...
M.Chapman Posted May 2 Share Posted May 2 (edited) 5 hours ago, Jeffrey Isaac Greenberg said: am going to assume DNG-enhanced-NR can be converted back to original DNG if ever required, so not saving (2) DNGs going forward... but no one confirmed it here yet.. I agree with MDM - it's not possible* to convert a de-noised DNG (which contains demosaiced and de-noised RGB data) back into the original DNG (which contained RAW sensor data). Best to store the original DNG + sidecar info. Several advantages to this. Less storage required Can still recreate denoised image if needed Can easily apply new/better denoise / image processing algorithms developed in future (e.g. HDR) * A small caveat to this. The DNG file spec allows both demosaiced and RAW sensor data to be stored in the same file for archival purposes. No idea if DXO DNGs do this (maybe that's why their DNG files are bigger?). But I haven't yet seen any commercial software that allows extraction of the archived RAW sensor data, even if it is there. Here's what ChatGPT has to say about it. Question - Can a DNG file contain both original RAW and demosaiced image data ChatGPT answer - No, a DNG (Digital Negative) file typically only contains one type of image data, either the original RAW data or the demosaiced (processed) image data. The purpose of the DNG format is to provide a standardized, lossless, and non-proprietary file format for storing raw image data, so it is usually not used to store both types of data in the same file. However, it is possible to embed the original RAW data within a DNG file as an option for archival purposes, but the demosaiced image data would generally be stored separately or converted from the raw data on-the-fly when needed. Mark Edited May 2 by M.Chapman Link to comment Share on other sites More sharing options...
MDM Posted May 2 Share Posted May 2 14 minutes ago, M.Chapman said: I agree with MDM - it's not possible* to convert a de-noised DNG (which contains demosaiced and de-noised RGB data) back into the original DNG (which contained RAW sensor data). Best to store the original DNG + sidecar info. Several advantages to this. Less storage required Can still recreate denoised image if needed Can easily apply new/better denoise / image processing algorithms developed in future (e.g. HDR) * A small caveat to this. The DNG file spec allows both demosaiced and RAW sensor data to be stored in the same file for archival purposes. No idea if DXO DNGs do this (maybe that's why their DNG files are bigger?). But I haven't yet seen any commercial software that allows extraction of the archived RAW sensor data, even if it is there. Mark There is no option to store the original raw in the enhanced DNGs produced by Denoise. Adobe made some major changes to the compression algorithm for the DNG files produced by Denoise since they first released it as the file sizes initally were enormous. They also made major changes to the compression algorithm for the DNG files produced by Photo Merge which are also way smaller than they used to be. The Denoise DNGs are described as lossless compressed and the Merge DNGs are described as lossy compressed although the quality of the images does not suggest anything being lost. Check out the Adobe forum for more detailed discussion if interested. 1 Link to comment Share on other sites More sharing options...
MDM Posted May 2 Share Posted May 2 (edited) 44 minutes ago, M.Chapman said: A quick look at file sizes shows some big differences (RAW / Adobe Denoise DNG / DXO PureRAW DNG) 46MP Nikon Z7 (91MB / 109MB / 199 MB) Not sure why the Nikon Z7 RAWs downloaded from DPReview is so large. Others I've seen are around 50-60MB. Mark They have most probably used the uncompressed NEF option which is pretty pointless for real world photography. I use the lossless compressed option and have done for years. Newer cameras have different NEF options. Edited May 2 by MDM Link to comment Share on other sites More sharing options...
BidC Posted May 2 Share Posted May 2 6 hours ago, Jeffrey Isaac Greenberg said: thanks for responses; space not an issue; currently have (3) 8TB Seagate external HDs as "identical" backups; 1.61TB free of 7.27TB & doubtful they'll ever fill; if they did, then upgrade to (3) 10TBs... I keep DNG, 8bitTIF27MB, & JPG27MB for each image; (TIF processing via 16bit full size, but not saved, too big at ~110MB ea.) am going to assume DNG-enhanced-NR can be converted back to original DNG if ever required, so not saving (2) DNGs going forward... but no one confirmed it here yet... I also have no idea why one would want DNG-NR ==> DNG, but... I'm kind of going 'off piste' here, but am wondering how you find the Seagate drives, JG ? I bought one recently (LaCie rugged), it caused problems with the laptop (high pitched beeping, and often the laptop wouldn't recognise it). I had to return it. The replacement did the same, so I am back to SanDisk (which seemingly aren't as resilient). Perhaps I should open a new thread (?) I'd be interested to hear what any others who have one have experienced. Link to comment Share on other sites More sharing options...
Julie Edwards Posted May 2 Share Posted May 2 (edited) In regards to file sizes.. the following links have good data and would apply equally to DxO and Adobe DNG's https://forum.dxo.com/t/why-are-dngs-so-much-larger-than-the-raw-files/27577/3 and https://forum.dxo.com/t/reduce-dng-files-size-when-exporting/32137/4 Goes into the Adobe Raw converter options... Bottom line - A raw image file contains single colour per pixel (colour filters if you know how the sensor works), a NR file is a demosaiced file containing RGB per pixel, hence the growth in size.. Edited May 2 by Julie Edwards Link to comment Share on other sites More sharing options...
MDM Posted May 2 Share Posted May 2 20 minutes ago, Julie Edwards said: In regards to file sizes.. the following links have good data and would apply equally to DxO and Adobe DNG's https://forum.dxo.com/t/why-are-dngs-so-much-larger-than-the-raw-files/27577/3 and https://forum.dxo.com/t/reduce-dng-files-size-when-exporting/32137/4 Goes into the Adobe Raw converter options... Bottom line - A raw image file contains single colour per pixel (colour filters if you know how the sensor works), a NR file is a demosaiced file containing RGB per pixel, hence the growth in size.. They are old links Julie. Adobe made some major changes to their DNG compression algorithms sometime summer or autumn 2023 which have greatly reduced DNG file sizes with no apparent loss of quality. This applies to the DNG files produced by Denoise as well as DNG files produced by HDR merges and panoramas. 3 Link to comment Share on other sites More sharing options...
Julie Edwards Posted May 2 Share Posted May 2 13 minutes ago, MDM said: They are old links Julie. Adobe made some major changes to their DNG compression algorithms sometime summer or autumn 2023 which have greatly reduced DNG file sizes with no apparent loss of quality. This applies to the DNG files produced by Denoise as well as DNG files produced by HDR merges and panoramas. Fair enough - but it does explain why a denoised (demosicaed) DNG could be larger than the original RAW (and the fact that a standard DNG is not demosiacied but a denoised one is...) Link to comment Share on other sites More sharing options...
Jeffrey Isaac Greenberg Posted May 3 Author Share Posted May 3 16 hours ago, M.Chapman said: I agree with MDM - it's not possible* to convert a de-noised DNG (which contains demosaiced and de-noised RGB data) back into the original DNG sidecars are easily lost -- disastrous in my past dealings... so setting all Adobe AI Denoise variables to 0 or minimum & then denoising a denoised DNG will not make 3rd DNG = 1st DNG...? how different will it be in appearance? can anyone suggest scenario(s) in which one saves ONLY denoised DNGs, deletes original DNGs, & faces crisis later? E.g., new AI Denoise advances requiring original DNGs, publisher requiring only original DNGs, etc.? I am still leaning towards NOT saving original DNGs ISO 1200-6400... Link to comment Share on other sites More sharing options...
Jeffrey Isaac Greenberg Posted May 3 Author Share Posted May 3 15 hours ago, BidC said: how you find the Seagate drives no emergencies for decade+ but over time, when I've heard silence turn to faint engine humming, I've replaced. always upgrading to latest... no single HD of dozen+ has lasted less than ~3 yrs & most 5+ years... 1 Link to comment Share on other sites More sharing options...
M.Chapman Posted May 3 Share Posted May 3 (edited) 7 hours ago, Jeffrey Isaac Greenberg said: sidecars are easily lost -- disastrous in my past dealings... Because you are working from a DNG in PS or LR, the ACR develop info can* be written directly into the DNG, without using a separate sidecar file that can be lost. I just tried the following; 1) Converted a RAW file from my camera into a DNG (using AdobeDNG converter) 2) Opened the DNG in PS (it opens in ACR) 3) Made some very obvious develop adjustments in ACR 4) Clicked Done, or click Open to send developed image to PS 5) At that moment the DNG file was automatically updated to include the develop settings I used and no separate sidecar was created. NB. The RAW image data in the DNG isn't altered, but the settings to be used to develop the RAW data are added as meta-data (in inbuilt sidecar) 6) I exited PS (without saving) 7) Reopened the DNG in ACR and my ACR develop settings (including a clone) are automatically applied. MDM may know more. Although I don't think either of us routinely use "RAW" DNGs. *There's a setting in ACR preferences that turns on saving sidecar info into the DNG. Mark Edited May 3 by M.Chapman Link to comment Share on other sites More sharing options...
Jeffrey Isaac Greenberg Posted May 3 Author Share Posted May 3 12 hours ago, M.Chapman said: 1) Converted a RAW file from my camera into a DNG (using AdobeDNG converter) 2) Opened the DNG in PS (it opens in ACR).................. MarkC me chum Thanks for these steps! Keeping for future reference. As I work thru me latest batch of (252) newbies am now finally aware I will keep original DNG, not DNG-NR because in my workflow DNG > DNG-NR > 16bitTIF-NR > 8bitTIF-NR > JPG-NR > submissions what I will keep is DNG, 8bitTIF-NR, JPG-NR knowing I can rework 8bitTIF-NR & maintain "smooth" histogram... but if day ever comes that an improved AI Denoise or other new function requires starting with original DNG, I will have it... Link to comment Share on other sites More sharing options...
M.Chapman Posted May 3 Share Posted May 3 9 minutes ago, Jeffrey Isaac Greenberg said: what I will keep is DNG, 8bitTIF-NR, JPG-NR knowing I can rework 8bitTIF-NR & maintain "smooth" histogram... Agree with keeping original DNG (which can have embedded xmp sidecar of your AR edits if you want) instead of DNG-NR But, IMO, reworking an 8 bit TIF-NR won't guarantee a smooth histogram if you make extensive brightness or saturation changes... I'd go back to the original DNG for that or you'll risk banding in skies etc., especially after NR where the concealing effect of a little noise won't be there to hide the banding. Mark Link to comment Share on other sites More sharing options...
IKuzmin Posted May 3 Share Posted May 3 2 hours ago, M.Chapman said: But, IMO, reworking an 8 bit TIF-NR won't guarantee a smooth histogram if you make extensive brightness or saturation changes... I'd go back to the original DNG for that or you'll risk banding in skies etc. I convert it to 16 bit just for making these adjustments, and then back to 8 bit. Link to comment Share on other sites More sharing options...
M.Chapman Posted May 4 Share Posted May 4 (edited) 7 hours ago, IKuzmin said: I convert it to 16 bit just for making these adjustments, and then back to 8 bit. ??? Once the data has been converted to 8 bit, data has been lost and won't be recovered by converting back to 16 bit. Mark Edited May 4 by M.Chapman 1 Link to comment Share on other sites More sharing options...
IKuzmin Posted May 4 Share Posted May 4 (edited) 6 hours ago, M.Chapman said: 13 hours ago, IKuzmin said: I convert it to 16 bit just for making these adjustments, and then back to 8 bit. ??? Once the data has been converted to 8 bit, data has been lost and won't be recovered by converting back to 16 bit. This is direct response to your suggestion above regarding histogram corruption when the extremes are adjusted on 8-bit images. This does not happen in my scenario. Nothing more. of course, no new data is generated by such conversion. I think that when it converts back, the clipped extreme data are replaced by the duplicated internal data, or something like this, at least the histogram remains smooth. Edited May 4 by IKuzmin Link to comment Share on other sites More sharing options...
MDM Posted May 4 Share Posted May 4 (edited) On 03/05/2024 at 01:13, Jeffrey Isaac Greenberg said: so setting all Adobe AI Denoise variables to 0 or minimum & then denoising a denoised DNG will not make 3rd DNG = 1st DNG...? how different will it be in appearance? can anyone suggest scenario(s) in which one saves ONLY denoised DNGs, deletes original DNGs, & faces crisis later? E.g., new AI Denoise advances requiring original DNGs, publisher requiring only original DNGs, etc.? I am still leaning towards NOT saving original DNGs ISO 1200-6400... Reverse Denoise - not possible - the second law of thermodynamics - entropy etc. Denoise doesn't create any accessible variables in the newly generated DNG apart from setting any existing manual NR back to zero. Resetting that has no effect and would be entirely pointless anyway. It's a destructive process and, as Julie point out, it is debayered and no longer raw. As well as the ones you mention, another scenario that I can envisage offhand where it might be a problem not keeping the original raw files would be for highly colour accurate work where one might wish to change the white balance at a later point but that is unlikely to be an issue for most users and it may not even be a problem at all. I've not tested it but presumaly somebody somewhere has done. That said, I keep the raw files and at present the generate DNGs as well Edited May 4 by MDM Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now