In science and technology, there has been a long and steady process of improving the accuracy of measurements of all kinds, along with similar efforts to improve image quality. The next goal is to reduce the uncertainty in the estimates that can be made, and inferences drawn, from the data (visual or otherwise) that have been collected. However uncertainty is never impossible. And since we must have, to a certain extent, there is much to be gained by accounting for as much uncertainty as possible.
In other words, we want to know how uncertain our uncertainty is.
This article was taken in a new study, led by Swami Sankaranarayanan, postdoc at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), and co-authors – Anastasios Angelopoulos and Stephen Bates of the University of California at Berkeley; Yaniv Romano of the Technion, Israel Institute of Technology; and Phillip Isola, assistant professor of electrical engineering and computer science at MIT. These researchers not only found accurate measurements of uncertainty, they also found a way to express uncertainty in a way that the layman could understand.
Their paper, presented in December at the Neural Information Processing Systems Conference in New Orleans, relates to computer vision – a field of artificial intelligence that involves training computers to extract information from digital images. The purpose of this study is for images that are slightly damaged or damaged (due to missing pixels), and in the process – computer algorithms, in particular – designed to reveal the portion of the signal that has been corrupted or otherwise obscured. This type of algorithm, Sankaranarayanan explains, “takes a blurry image as an input and gives you a clean image as an output”—a process that takes place slowly.
First, there is the encoder, a type of neural network specifically trained by researchers to do the job of removing the blurring of fuzzy images. An encoder takes a negative image and, from it, creates a virtual (or “hidden”) representation of the white image in a format – consisting of a series of numbers – that makes sense on a computer but may not make sense to most people. . The next step is the decoder, which has several models, which are usually neural networks. Sankaranarayanan and his colleagues worked with a decoder called the “generative” model. In particular, they used an off-the-shelf model called StyleGAN, which takes numbers from a captured image (of a cat, for example) as its input and then creates a full, refined image (of the cat). So the whole process, including the encoding and decoding stages, results in a richer picture from the originally muddy translation.
But how much faith can one place in the accuracy of the following image? And, as discussed in the December 2022 paper, what is the best way to show uncertainty in this picture? A common technique is to create a “reality map,” which displays a value — somewhere between 0 and 1 — indicating how confident the model is in the accuracy of each pixel, taken one by one. This method has a problem, according to Sankaranarayanan, “because the prediction is made independently for each pixel. But meaningful things happen in groups of pixels, not within a single pixel,” he adds, which is why he and his colleagues are developing a completely different way of looking at uncertainty .
Their method is based on the “semantic properties” of the image – groups of pixels that, when taken together, have meaning, forming the face of a person, for example, a dog, or some other recognizable object. The goal, Sankaranarayanan asserts, “is to estimate uncertainty in a way that involves groups of pixels that people can easily interpret.”
While a well-known method can provide a single image, making a “good guess” about what the actual image should be, the uncertainty of the image is often difficult to detect. The new paper argues that for real-world applications, uncertainty must be expressed in a way that makes sense to people who are not experts in machine learning. Instead of creating a single image, the authors have developed a method of creating multiple images – each of which can be correct. Also, they can set real limits on the range, or sub-range, and provide a reasonable guarantee that the real image is somewhere in between. A lower version may be given if the user is comfortable with, say, 90 percent certainty, and a lower version if the risk is acceptable.
The authors believe that their paper provides the first algorithm, designed for the production model, which can establish the uncertainty intervals associated with the meaning (defined in the abstract) of the image and come up with “quantificational certainty.” Although this is a very important event, Sankaranarayanan sees it as just a step towards reaching the “greater goal.” So far, we have been able to do this for simple things, such as restoring images of human or animal faces, but we want to extend this method in more complex areas, such as medical imaging, where our ‘quantity assurance’ would be very important. .”
Suppose the film, or radiograph, of the chest X-ray is distorted, he adds, “and you want to recreate the image.” When you are given several images, you want to know that the real image is within this range, so you don’t need anything complicated” – information that can reveal whether a patient has lung cancer or pneumonia. In fact, Sankaranarayanan and his colleagues have already started working with radiologists to see if their pneumonia prediction system could be clinically useful.
Their work could also be important in the legal field, he says. “The image from a surveillance camera can be blurry, and you want to add to that. Examples of how to do that already exist, but it’s not easy to measure uncertainty. And you don’t want to make a life-or-death mistake.” The tools he and his friends are developing can help identify a guilty person and free an innocent person.
Much of what we do and much of what is happening in the world around us is questionable, writes Sankaranarayanan. Therefore, a better understanding of that doubt can help us in many ways. One reason is that it can tell us a lot about things we don’t know.
Angelopoulos was supported by the National Science Foundation. Bates was supported by the Foundations of Data Science Institute and the Simons Institute. Romano was supported by the Israel Science Foundation and a Career Advancement Fellowship from the Technion. Sankaranarayanan and Isola’s research for this project was supported by the US Air Force Research Laboratory and the US Air Force Artificial Intelligence Accelerator and was carried out under Cooperative Agreement Number FA8750-19-2-1000. MIT SuperCloud and Lincoln Laboratory Supercomputing Center also provided the computer. resources that contributed to the results reported in this work.
