KARWY (pronounced car-we) is a free program, soon to be fully open source, that credibly repairs visible artifacts in Sony ARW2 raw files (note: KARWY cannot process DNGs already converted from ARW2 files) and wraps the resulting raw data in an uncompressed DNG. This static page is the primary entry to KARWY, and will always provide links to any released versions. The current version uses a WWW form interface at:
http://super.ece.engr.uky.edu:8088/cgi-bin/karwy.cgi
Originally, this software was considered a "Beta test" version (and bugs and fixes since posting are noted later on this page), and the plan was to release a stand-alone version when it became stable. Well, it's stable enough, but it is built for a Linux environment using Adobe DNG Converter via wine, which makes it a bit of a nightmare to install. Thus, I now plan to continue to maintain the CGI interface version as the primary way for folks to use KARWY (welcome to "the cloud" -- i.e., running KARWY on my computer, not yours). Although I'll happily share source code with anyone who asks, I'm not trying to package it as a stand-alone version (yet). A research paper detailing how KARWY works was published at the IS&T Electronic Imaging 2016 conference in February 2016; Sony ARW2 compression: Artifacts and credible repair (PDF slides, published paper).
Sony has used a variety of raw image formats in their cameras over the years. One format, ARW2, uses lossy compression which efficiently maps 14-bit samples into 11/7-bit data. Unfortunately, artifacts (e.g., local posterization) can become significant in postprocessing. Affected camera models include:
It is quite rare that ARW2 artifacts are visible in well-exposed, normally-processed, images. In general, artifacts are most often found in images of contrasty scenes where very bright pixels are horizontal neighbors of very dark pixels and postprocessing has significantly boosted the darker tones. The artifacts are generally more severe when the camera can deliver a very large dynamic range (e.g., better sensors at low ISO) or a high-resolution lens is used on a sensor without an AA (anti-alias) filter. We call the most disturbing artifacts "Blondie artifacts" -- parallel lines running horizontally near high-contrast edges. There are also posterization artifacts with less obvious structure. It is interesting to note that some of this artifacting is arguably beneficial, slightly increasing apparent sharpness of the image.
Professor Hank Dietz and his Aggregate.Org research group (based in the University of Kentucky) developed KARWY to perform credible repair of the ARW2 data. Repairs are not restricted to where the artifacts are most obvious, but attempt to enhance all areas of the image where there is any ambiguity about pixel values. The tool works by modeling uncertainty of each pixel value and then applying several algorithms, especially texture synthesis, to obtain better estimates of pixel values.
KARWY's processing can reduce artifacting and increase signal to noise ratio, but the improvements made should not be assumed to be forensically valid. Things may look better, but actually be slightly less like the scene photographed. For this reason, it is suggested that the original ARW2 file should still be archived. KARWY does offer the option of embedding the ARW2 in the repaired DNG so that the original image data could be recovered from the DNG, but the resulting DNG file is much larger. For achiving, We recommend compressing image files using a standard lossless file compression utility, for example, xz.
The WWW form interface for, and image processing in, KARWY is implemented using custom C code written by Hank Dietz, but the following tools are used in decoding/encoding image files and implementing the CGI interface: DCRaw, Adobe DNG Converter, ExifTool, ImageMagick, XZ, and CGIC.
The only thing set in stone is our name.