Camera Makers Defend Proprietary RAW Formats Despite Open Standard Alternative (theverge.com) 65
Camera manufacturers continue to use different proprietary RAW file formats despite the 20-year existence of Adobe's open-source DNG (Digital Negative) format, creating ongoing compatibility challenges for photographers and software developers.
Major manufacturers including Sony, Canon, and Panasonic defended their proprietary formats as necessary for maintaining control over image processing. Sony's product team told The Verge their ARW format allows them "to maximize performance based on device characteristics such as the image sensor and image processing engine." Canon similarly claims proprietary formats enable "optimum processing during image development."
The Verge argues that this fragmentation forces editing software to specifically support each manufacturer's format and every new camera model -- creating delays for early adopters when new cameras launch. Each new device requires "measuring sensor characteristics such as color and noise," said Adobe's Eric Chan.
For what it's worth, smaller manufacturers like Ricoh, Leica, and Sigma have adopted DNG, which streamlines workflow by containing metadata directly within a single file rather than requiring separate XMP sidecar files.
Major manufacturers including Sony, Canon, and Panasonic defended their proprietary formats as necessary for maintaining control over image processing. Sony's product team told The Verge their ARW format allows them "to maximize performance based on device characteristics such as the image sensor and image processing engine." Canon similarly claims proprietary formats enable "optimum processing during image development."
The Verge argues that this fragmentation forces editing software to specifically support each manufacturer's format and every new camera model -- creating delays for early adopters when new cameras launch. Each new device requires "measuring sensor characteristics such as color and noise," said Adobe's Eric Chan.
For what it's worth, smaller manufacturers like Ricoh, Leica, and Sigma have adopted DNG, which streamlines workflow by containing metadata directly within a single file rather than requiring separate XMP sidecar files.
thanks for the info (Score:2)
it makes no difference (Score:4, Insightful)
The big player camera formats are widely supported, because they are de facto standards. Just like WK1 was a widely supported format. The reason the small players have to use Adobe's format is if they didn't, no software would support their cameras.
Obviously Adobe would prefer to not have to bother supporting manufacturer-specific formats but if the change did happen, that would just mean Adobe would drop support for the manufacturer-specific formats and you wouldn't be able to use old cameras with current versions of Photoshop or whatever. There's no world where Canon et al. adopting DNG is better for you.
Re:it makes no difference (Score:5, Insightful)
The big player camera formats are widely supported, because they are de facto standards.
Eventually, two years after any major format change. Until then, they are supported only by a handful of major players like Adobe who work with the camera manufacturer under an NDA to develop support for those formats, and it still takes additional years before they are fully supported even when you limit yourself to software by the major players.
There's no world where Canon et al. adopting DNG is better for you.
I can't speak to the DNG format itself specifically, but there is no world in which the camera companies being forced to provide full open specifications for their data files is not better for you. Whether that is in the form of using the DNG format with full documentation of any extensions to that format well in advance of any hardware release, in some new format developed by a standards body under similar terms, or in a proprietary format with a fully functional C library for working with it that is provided by the camera vendor under a permissive open source license is an implementation detail.
Re: (Score:2)
Eventually, two years after any major format change.
Great, I don't buy them new anyway. And with practically all the photography going mirrorless, there's no good reason not to buy used since the most fragile part of the camera doesn't exist.
Re: (Score:2)
Eventually, two years after any major format change. Until then, they are supported only by a handful of major players like Adobe who work with the camera manufacturer under an NDA to develop support for those formats, and it still takes additional years before they are fully supported even when you limit yourself to software by the major players.
I am pretty sure support in FOSS is faster than that, I use darktable on Linux.
Cost and patents on the encoding chips (Score:2)
This needs to be looked at the total chain of acquiring and image from the sensor, encoding it, saving it to a flash memory card on the camera, transferring it to a computer, opening and decoding the file and editing it.
Each of the stages needs to be looked at for patents, vendor specific hardware chips, cost/efficiency/electricity used and form factor.
For example, forcing a camera manufacturer to redesign its hardware to fit a new processing chip.
Adobe Tax (Score:2, Insightful)
I don't need to ready anything in the article, and could be entirely wrong - but there is no chance in hell I would tie my hardware business to Adobe license/tax hell. Even if its 'open' according to them. Without knowing anything more I know this is a way for them to achieve a lock-in of some sort so they can force their will on manufactures.
There is exactly 0 chance that any of this was done for the good of the industry. Its Adobe, they want to be the industry by themselves.
Re: (Score:2)
Re:Adobe Tax (Score:4, Insightful)
If you're not interested in RAW files you probably shouldn't be in the market for an expensive camera.
Re: (Score:3)
All digital cameras can save to JPG. Most do so by default.
Re:Adobe Tax (Score:5, Informative)
All digital cameras can save to JPG. Most do so by default.
This isn’t about whether a camera can save a JPEG — of course it can, and most do by default. But JPEG is a delivery format, not a capture format. It’s lossy, baked-in, and optimized for final output — not for post-processing. Professionals shoot in RAW because it preserves the full sensor data for editing, grading, and retouching. The problem isn't that JPEG exists — it's that the RAW files needed for that editing process are fractured across dozens of proprietary formats, each with varying degrees of third-party support. That’s the real issue: a fragmented ecosystem that complicates the critical early stages of the professional workflow, not the existence of a lossy shortcut for casual use.
Re: (Score:2)
Re: Adobe Tax (Score:2)
Re: (Score:1)
You clearly don't know the benefits of raw files.
Re: (Score:2)
i would not buy an expensive camera unless it saves the photos in a file format that Linux & gimp can handle like jpg or png, i would also recommend staying away from google's wonky file format too like webm and webp
These cameras all save in open formats like jpeg as well, and you can also get them to save both. RAW files are (at least initially) pretty much sensor dumps - the data isn't lossily compressed or processed (or at least minimially processed). This typically allows you do processing yourself before you "develop" a jpeg - things like tinker with the exposure, noise reduction etc in the image, or parts of it. If you're looking for an open source competitor to Lightroom and Capture One, darktable [darktable.org] is more up th
Re: (Score:2)
Re: (Score:3)
Leaving aside that the idea that jpeg or png are a substitute for raw, Gimp isn't the tool to handle raw files, it has the same support for RAW that Photoshop does: i.e. a separate tool is used (in the Adobe world it used to be a whole separate program called Adobe CameraRAW) to import the files. In the modern workflow you use Adobe Lightroom to run RAW files and with a click of a button it pushes over to Photoshop using DNG / TIFF.
In the Gimp world you would do the same thing by opening the RAW file in Dar
Re: (Score:1)
For most cameras, the files get converted automatically to *.jpg
Which is pretty useless for any postprocessing of "real photos".
If your phone is not good enough for you, buy a cheap camera with an SD card slot, or USB port, and be done with it.
Duh (Score:5, Insightful)
"Camera manufacturers continue to use different proprietary RAW file formats despite the 20-year existence of Adobe's open-source DNG (Digital Negative) format, creating ongoing compatibility challenges for photographers and software developers."
Because transitioning offers nothing to those manufacturers or their customers. DNG was created by Adobe because it was in Adobe's interest. Referring to it as "open-source" is just a ploy to stir up a base, photographers using RAW formats are not concerned with open source, they are concerned with quality and support. Those "proprietary RAW file formats" provide quality and support.
Re: (Score:2)
The advantage to photographers is that they can use the software of their choice to work with the RAW files. There are a number of open source apps, my favourite being the awkwardly named Another Raw Threapee (ART), as well as commercial ones, which support the DNG format.
One thing TFA doesn't mention is that a lot of phones support it too. Google Pixel phones produce DNG files when told to save RAWs.
Re: (Score:1)
Photographers are already using their software of choice to work with raw formats. Often, that software takes advantage of specific hardware and software features of the $$,$$$ to $$$,$$$ cameras and associated equipment that created the raw files. Staying within the family for software to work with digital negatives and images is not a big jump when someone has already invested in a single platform of camera bodies and lenses, and most photography software that is worth using in the long run provides timel
Re:Duh (Score:5, Interesting)
Photographers are already using their software of choice to work with raw formats.
Eventually, yes. The real problem is that every time Canon or Nikon updates their file format, it takes almost two years for software like libraw to support it. For example, the CR3 file format came out in March of 2018. Libraw got support in October of 2019. So there was a 19 to 20 month period in which photographers were NOT able to use the software of their choice to work with that raw format.
On the flip side, having their own format means never having to say "I'm sorry". When Canon added support for dual-pixel sensors, they didn't have to get anyone to agree to their format. They just did it. And there are other advantages to that, particularly when it comes to not disclosing features like that to other companies ahead of the release date.
But back on the negative side, that means nobody outside of Canon got to look at that format ahead of time, and Canon's format absolutely sucks for efficiency. The dual-pixel images take up twice as much space as standard CR2 files. Had they gotten more eyes on it, someone would have pointed out that they could save the second slice as a difference signal relative to the original slice, and by doing so, turn most of the image into long runs of zeroes, then do run-length encoding, and massively reduce that overhead.
And that could mean as much as a 2x increase in the number of shots that their cameras can shoot at maximum speed before the RAM buffer runs out, so Canon's decision to use a proprietary standard didn't just hurt photographers' pocketbooks by making them buy more flash cards. It also hurt the usability of their cameras in a very direct way.
I don't mind if these companies want to have their own formats as long as they publish them under a BSD license on a regular basis. And ideally, they should do so a bit *before* the release of hardware that depends on them, so that the broader community can point out obvious flaws in their data format and push for improvements before it is permanently baked into hardware. But even if they don't, an open-source-licensed library would allow others to make backwards-compatible changes to enable a second, improved version of the format gated behind a flag so that they could flip the bit in a future revision of the firmware and get the benefits; that's the nice thing about being able to update the firmware. :-)
But the current situation absolutely sucks for photographers, and it is entirely because companies like Canon and Nikon do not support open standards AND do not provide permissively licensed open source libraries for their own proprietary standards. I really don't care which one they do, but I'm really disinclined to buy their hardware until it has broad support in the open source community, and I know other folks are as well, so not doing so absolutely does cost them sales.
When professional photographers' time is worth $$$ to $,$$$ per hour, the fact that the software or standard is open source might not matter to their teams' workflows or bottom lines.
Here's the flaw in your thinking. For every one professional photographer that buys these cameras, there are probably a hundred advanced hobbyists who want something better than an iPhone, and that might even be an underestimate. Yes, if Canon and Nikon want to lose a huge chunk of their sales for a couple of years after every major format change, then they can feel free to ignore the open source world, but if you think it doesn't have a real negative impact on the first two years of sales every time they make an incompatible change, you're kidding yourself.
Worse, that drop in sales is usually a permanent loss of sales. By the time the format is supported, they have usually released a new camera that uses it, and there's a decent chance that people will buy that instead of the previous camera, at which point someone has now skipped an entire generation o
Re: (Score:2)
RAW file formats, because they're raw, reflect the hardware. When the hardware changes, the raw data it provides changes. DXF, which is a container format, doesn't do anything about that, because it can't. Modern Canon RAW files are TIFF, which is a much better choice than Adobe's DXF, but you still have to update your software to support new hardware.
If your software of choice isn't doing that in a timely manner maybe you should consider something else?
Re: (Score:2)
RAW formats are typically very close to the hardware. Ideally they're the bits that come out of the ADC, plus a little bit of metadata to identify where they came from. That does vary somewhat by manufacturer. Some like to hide their sins more than others.
Re: (Score:2)
RAW formats are typically very close to the hardware. Ideally they're the bits that come out of the ADC, plus a little bit of metadata to identify where they came from. That does vary somewhat by manufacturer. Some like to hide their sins more than others.
Like dust deletion. Yeah.
I would suggest amending that statement slightly, though. It should be possible to *obtain* the exact bits that come out of the ADC by reading the raw file. However, a raw file should not *be* the exact bits. Ideally, you want the raw bits to be compressed as much as humanly possible in a lossless fashion. Otherwise, you're wasting flash storage and you're limiting how many photos you can shoot before your camera starts to slow down because of the write speed limit on the flash
Re: (Score:2)
> So there was a 19 to 20 month period in which photographers were NOT able to use the software of their choice to work with that raw format.
you mean those who actually use open sourced Libraw ... because others or those who paid did not wait that long ... commercial apps like ACR/LR, C1, DxO PhotoLab, etc were prompt in support ... freeriders / those waiting for public releases of libraw, etc / had to wait that is true :-) ... and free Adobe DNG converter was converting CR3 to DNG for freeriders...
Yes, I'm aware of that. But this is in the context of a thread about open source specifically.
Re: (Score:2)
Actually, there's a program called "dcraw" that is an open-source RAW converter. It's permissive license and is basically the core RAW file importer for practically every third party photo program out there.
The camera manufacturers make their own RAW processing software (and release it for free), but dcraw was used because it was easier to license into programs and could offer more tweaks than what the manufacturers offered. Plus, it works across multiple operating systems - so your digital camera which mig
Re: (Score:1)
> Actually, there's a program called "dcraw"
no longer active for years... that is btw one of many different reasons why libraw was born
Re: (Score:2)
...they are concerned with quality and support.
Dumping the sensor data into a standardized format does nothing to reduce quality and support. The worst that will happen is that quality sees no change, and support increases.
A little misleading, a little true. (Score:5, Insightful)
It's more complex than the article suggests.
Somewhat ironically, the problem DNG proports to solve is a problem the format itself experiences. Yes, it is true that the camera manufacturers update their image formats and it takes time for companies to catch up. But at the same time the DNG format is on it's 7th iteration, if your camera is using the 2023 version of DNG but your software only supports up to the 2021 version of DNG, it's exactly the same problem as if you've got a 2023 version from your Canon camera but your software only supports up to 2021 version.
Plus as a container format, anybody can put whatever they want in the file and you still need the matching codec for that piece of the content. In many ways it's like so many other audio and video formats, the file can be opened but the specific codec is still required.
Re:A little misleading, a little true. (Score:4, Insightful)
Thank you. I suspected something similar to this.
The DNG then somewhat becomes a stumbling block as you can't even be sure if software that handles it can handle it.
The nature of a raw file being a dump from the sensors means that it will be sensor specific. I'm sure there are some perks to DNG being a single file, and probably some standardization of metadata fields, and a singular metadata format to parse, but I imagine there's a lot that is different camera to camera and brand to brand by nature of how the device itself is designed.
Re: A little misleading, a little true. (Score:2)
Re: (Score:2)
Somewhat ironically, the problem DNG proports to solve is a problem the format itself experiences. Yes, it is true that the camera manufacturers update their image formats and it takes time for companies to catch up. But at the same time the DNG format is on it's 7th iteration, if your camera is using the 2023 version of DNG but your software only supports up to the 2021 version of DNG, it's exactly the same problem as if you've got a 2023 version from your Canon camera but your software only supports up to 2021 version.
Yeah, and that's a problem with standards in general unless they are backed by an official open source reference implementation under a permissive license.
An ideal standards body would have a policy whereby if you want to change or improve the standard, you start with a problem statement, followed by a proposed solution, folks debate whether the solution makes sense, and the process ends with you submitting a change to the reference implementation. And that means as soon as the standards change, any softwa
Re: (Score:2)
Canon's RAW format uses a TIFF or Quicktime container. A much better choice than DXF. Still a container though, if you want access to the raw data you need to update your software whenever the hardware that produces it changes.
Maybe that's why it's "raw"? (Score:2)
A $500+ camera problem? (Score:2)
This is only a problem for people with $500+ cameras AND feel the need to tweak their pictures beyond cropping out exes and rando bystanders. The 99% of us who are happy with pics from our phones don't really care.
No one you loved ever had a wedding? (Score:2)
This is only a problem for people with $500+ cameras AND feel the need to tweak their pictures beyond cropping out exes and rando bystanders. The 99% of us who are happy with pics from our phones don't really care.
Ever hire a professional photographer for a wedding, corporate event, bar mitvah, baptism, etc? Also phones use RAW now too because it's easier to fix your mistakes, like white balance issues...ever take a pic and everyone looks like they've been soaked in piss because the homeowner thought warm 2700k lightbulbs were a good idea? Your eyes can adjust, but all digital sensors are still inferior to a human eye.
With RAW, you have a lot more ability to de-piss the photo and make whites look white instead
Re: (Score:2)
Phones seem to handle the white balance just fine. Though I'd be lying if I said my wife or I busted out the Nikon to take pics inside of the house.
Re: (Score:2)
Ever hire a professional photographer for a wedding, corporate event, bar mitvah, baptism, etc?
The professional photographer has professional software, and can load the various RAW files from their various professional cameras.
Re: (Score:2)
Thanks for proving my point. Those "professional photographer"s are the 1% I was talking about. I didn't say it wasn't a problem for everyone. But the rest of us are fine with what a phone offers.
DNGs are not standard + don't trust Adobe (Score:5, Insightful)
1. Your edits are typically lost if you open with another program.
2. There's no guarantee you won't lose your edits in the distant future if Adobe changes their software.
3. There's no option in Adobe's software to force a final edit. So if you crop out 3/4 of the photo, your file size is still the same, wasting all those bytes...when, let's be real...you won't edit this 10 years down the road, but now you're paying Adobe's exorbitant cloud storage fees to store this file. Additionally, if there was a final edit, it would guarantee your changes are permanent and viewable across platforms or even future versions of lightroom
4. Adobe doesn't even support DNG very well any more. Their latest Lightroom Cloud version buries that option and tries to force you save RAW
I have 10 years of family photos in DNG. I regret that decision deeply. I converted them all to JPG and have never been happier. I will never make that mistake again. Adobe is probably the worst software company out there. They're very anti-consumer and Lightroom was a massive debacle.
For those who don't know. They had a decent program for 20 years ago named Lightroom. However, it BREAKS and corrupts data if opened from another location, so you can't use it with Dropbox between your desktop and laptop, for example. This is because it internally uses a database to store your edits that writes data on close. Also, since it writes the edits in a DB, you don't see them unless you export. It also has shockingly horrid performance. So they create a cloud-native version. They name it Lightroom and the original one "Lightroom classic"...which is the dumbest decision in the history of software.
OK, so their new Lightroom (let's call it LR Cloud) SUUUUUUCKS. It has about 1/10th of the features, but it performs well and works cloud natively and they charge a monthly fee to use it. The fee is reasonable until you realize it doesn't support local storage. You can export, but until recently, not really read from local storage. So now you have a dead product they barely support and forces you to use a single machine. You have a cloud native one that until 10 years after it was released, was largely useless.
And the worst part? Because the new product is named "Lightroom" everytime you google something, like "Lightroom tethering," it will open a 15 year old blog post explaining how to do it on the old version, but not the new.
How badly do you want to depend on this company to steward the ability for software to read photos of your children? DNG was a disaster and can't be trusted. Adobe is one of the least trusted names in software by their users. Being a Lightroom customer is a lot like being an Oracle RDBMS user...you use it because you're afraid of the competitors with your valuable data, but you hate every step of it and curse it and think "There's got to be a better way" every time....but people are very risk averse with their photos and their data...so it's a profitable product.
Finally, RAW formats are very specific to manufacturers and their hardware. Canon stores dual pixel info, for example, that can slightly alter focus. There are a lot of minor proprietary features that are here today and gone tomorrow and will vary widely by model. Standardizing that would be a disaster. Camera bodies are not a commodity to standardize. They're expensive professional instruments that experts tweak to eeeek out the tiniest of performance benefits or useful features. The manufacturers have good reasons to avoid anything from Adobe, especially DNG and the users would hate the result.
Want to fix the problem the author is complaining about? How about a good
Re: (Score:1)
I use DxO Photolab (DxO Optics), and it uses sidecar files along the raw files. The other day i opened up a picture i edited with an old version of dxo, and it took the sidecar perfectly.
Capture NX / NX Studio uses sidecar files too.
Re: (Score:1)
2: The customer wants usable images. And 99% of the time the do not want a raw file. They want a final product. I've sent HQ Jpeg's, png's and even very big lossless compressed TIFF files. You can't tell one from the other even at pixel level.
Re: (Score:2)
What the customer wants is irrelevant. They aren't dealing with the workflow from the camera, you are. You have to store and work with these formats, and it is a bit of a mess.
Re: (Score:3)
1. Your edits are typically lost if you open with another program.
This has nothing to do with DNG. This is because Lightroom is a nondestructive editor. It is nondestructive for JPEG images, too, BTW.
2. There's no guarantee you won't lose your edits in the distant future if Adobe changes their software.
Same response.
3. There's no option in Adobe's software to force a final edit. So if you crop out 3/4 of the photo, your file size is still the same, wasting all those bytes...when, let's be real...you won't edit this 10 years down the road, but now you're paying Adobe's exorbitant cloud storage fees to store this file. Additionally, if there was a final edit, it would guarantee your changes are permanent and viewable across platforms or even future versions of lightroom
There actually is. You export it as a JPEG and then delete the original and import the JPEG. The reason almost nobody does this is that you lose quality every time you compress an image, and if you start out with the modified image and want to make further changes, that quality loss is compounded. So if you might ever want to make any future edits, you're g
Simple answer TIFFs are too big (Score:2)
Is there something wrong with TIFF that I don't know about? I mean yes, we could probably do better than LZW compression these days, but there's no reason someone can't extend that format with a new compression algorithm and propose it to the standards body as long as they don't modify the container structure itself, and let's face it, the container itself isn't a significant percentage of the size of the file. :-)
The file sizes are huge and no one has extended it and if they did, it would need widesprea
Re: (Score:2)
Also, while you're poking holes in my grievances about DNG implementation details, the fact remains...DNGs are not reliable. You save a DNG in lightroom, it won't look the same in another program. Is this the fault of the spec or Lightroom?
Neither. It is inherent in the nature of RAW files. You don't have pixels. You have a bunch of subpixels, and different companies use different approaches for computing the color value of a pixel based on the geometry of where those subpixels are located relative to where the pixel ends up being. And there are different approaches for doing color conversion from the native gamut of the image sensor to the native gamut of your display, not to mention differences in how they do gamma conversion, etc., and
JPGs are consistent (Score:2)
Re: (Score:2)
Look...you want to talk about Gamut and stuff...that's pretty esoteric. Yes, things look different on different monitors, but JPGs look the same on Windows vs Mac, by default. Saying otherwise is really splitting hairs and doesn't match the typical experience.
To be fair, that's only recently. Up until a few years ago, the standard gamma for Macs was different from Windows, and JPEG files looked quite different.
However, the point made is that eventually, a final image is created and the only lossless output format that is universally supported is TIFF.
And PNG, which also supports up to 48-bit color. It's probably a coin toss between the two; the only compression algorithm PNG uses is also supported by TIFF.
The big problem is that lossless compression hasn't made a lot of progress since the advent of Burrows-Wheeler (which, sadly, is still not used for image compression by anyone, AFAIK; if anybody wan
Re: DNGs are not standard + don't trust Adobe (Score:2)
Then what is a RAW file? (Score:2)
Hilarious that you are sitting at +4 because you wrote a novel-length rant against a Slashdot-favorite target in Adobe, when in reality, every single complaint of yours stems from your ignorance of a) what a RAW file is, and b) that A DNG is a RAW file.
Wanna put some effort into explaining yourself? You make a vague statement with no justification and you expect people to fawn over you? Unless you put effort into explaining something that isn't common knowledge, you're just lazily trolling.
uh huh, sure (Score:2)
Nope (Score:4, Informative)
All my experience with DNG has been bad. All my experience with Adobe has been bad. Why not combine two negatives and make a positive? Sure.
When Raw isn't Raw (Score:3)
I run away ... (Score:2)
There's a really big question unanswered here (Score:2)
They say DNG is "open" - who controls the format's definition? As far as I can tell, Adobe is tightly holding onto it with its grubby hands, and "open" just means "we're letting you use it free".
If that's indeed the case, if I have to choose a single corporation to trust - personally I'd pick Nikon (or Canon, if you're in their hardware camp).
Re: (Score:1)
Fuck Adobe (Score:1)
Somewhat Annoying (Score:2)
Nikon has ".NEF" affectionately known as "Nobody Else's Format" so you have to go chase a software package for it and install it on your computer to make it provide thumbnails and the like. Otherwise, Nikon software tools and the most common editing tools all have it and have for years. But at this point, Nikon changing would probably be a bad thing for all its customers all the way back thru digital time. Better to leave the cameras alone, let them chuck out ".NEF" forever, and just make a software pac