Analyze images at a glance and track the life cycle of snow – TechCrunch

The research in the area of ​​machine learning and AI, now a key technology in virtually every industry and business, is far too voluminous for anyone to read it all. This column, Perceptron (formerly Deep Science), aims to bring together some of the most relevant recent discoveries and papers – particularly, but not limited to, artificial intelligence – and explain why they matter.

This month in AI, engineers at Penn State announcement that they have created a chip capable of processing and classifying nearly two billion images per second. Carnegie Mellon, meanwhile, has sign a $10.5 million contract with the US military to expand its use of AI in predictive maintenance. And at UC Berkeley, a team of scientists is applying AI research to solve climate problems, like agreement snow as a water resource.

Penn State’s work aimed to overcome the limitations of traditional processors when applied to AI workloads, by specifically recognizing and classifying images or objects within them. Before a machine learning system can process an image, it must be captured by a camera’s image sensor (assuming it is a real-world image), converted by the sensor from light signals to electrical signals and then converted back to binary data. Only then can the system “understand” the image sufficiently to process, analyze and classify it.

Penn State engineers, including postdoctoral fellow Farshid Ashtiani, graduate student Alexander J. Geers, and associate professor of electrical and systems engineering Firooz Aflatouni, have devised a workaround that they say removes the worst aspects. time-consuming traditional on-chip AI image processing. Their custom 9.3 square millimeter processor directly processes light received from an “object of interest” using what they call an “optical deep neural network”.

3D render, computer board with circuits and chip. Picture credits: Xia Yuan/Getty Images

Essentially, the researchers’ processor uses “optical neurons” interconnected using optical wires, called waveguides, to form a multi-layered deep network. Information passes through the layers, with each step helping to classify the input image into one of its learned categories. Thanks to the chip’s ability to calculate when light travels through it to directly read and process optical signals, the researchers say the chip does not need to store information and can perform image classification. complete in about half a nanosecond.

“We are not the first to deliver technology that reads optical signals directly,” Geers said in a statement, “but we are the first to create the complete system in a chip that is both compatible with existing technology and scalable to work with more complex data.He expects the work to have applications for automatically detecting text in photos, helping self-driving cars recognize obstacles and other vision-related tasks. computer.

At Carnegie Mellon, the college’s Auton Lab focuses on a different set of use cases: applying predictive maintenance techniques to everything from ground vehicles to power generators. Supported by the aforementioned contract, Auton Lab Director Artur Dubrawski will lead a fundamental research effort to broaden the applicability of computational models of complex physical systems, known as digital twins, to many fields.

Digital twin technologies are not new. GEAWS and other companies offer products that enable customers to model digital twins of machines. Based in London SensSat creates digital twin models of locations for construction, mining and energy projects. Meanwhile, startups like Lacuna and Nexar are building digital twins of entire cities.

But digital twin technologies share the same limitations, the main one being inaccurate modeling from inaccurate data. As elsewhere, it’s trash inside, trash outside.

To address this and other barriers to wider use of digital twins, Dubrawski’s team is collaborating with a range of stakeholders, such as critical care clinicians, to explore scenarios, including in the area of health. The Auton Lab aims to develop new and more efficient methods of “capturing human expertise” so that AI systems can understand poorly represented contexts in data, as well as methods of sharing this expertise with users.

One thing AI may soon have that some people seem to lack is common sense. DARPA has funded a number of initiatives in different labs that aim to give robots a general idea of ​​what to do if things aren’t going well when they’re walking, carrying something, or grabbing an object.

Usually, these models are quite fragile, failing miserably as soon as certain parameters are exceeded or unexpected events occur. By training “common sense” in it, they will be more flexible, with a general idea of ​​how to salvage a situation. These aren’t high-level concepts, just smarter ways to deal with them. For example, if something falls outside the expected parameters, it can adjust other parameters to counter it even if they are not specifically designed to do so.

This doesn’t mean the bots will improvise everything – they just won’t fail as easily or as hard as they currently do. Current research shows that locomotion over rough terrain is better, moving loads are better carried, and unknown objects are better grasped when “common sense” training is included.

The UC Berkeley research team, on the other hand, is focusing on one area in particular: climate change. The Berkeley AI Research Climate Initiative (BAIR) – which was launched recently, organized by computer science PhD students Colorado Reed and Medhini Narasimhan and computer science PhD student Ritwik Gupta – is seeking partners among climate experts, government agencies and the industry to achieve meaningful climate and AI goals.

One of the first projects the initiative plans to tackle will use an AI technique to combine measurements from aerial snow observations and freely available weather and satellite data sources. AI will help track the life cycle of snow, which is currently not possible without a lot of effort, allowing researchers to estimate and predict the amount of water in snow in the mountains of the Sierra Nevada – and to predict the impact on the region’s flow.

A press release detailing BAIR’s efforts notes that the snow condition has an impact on public health and the economy. Around 1.2 billion people around the world depend on melting snow for water consumption or other purposes, and the Sierra Mountains alone provide water for more than half of the California people.

Any technology or research done by the climate initiative will be openly published and will not be licensed exclusively, said Trevor Darrell, co-founding director of BAIR and professor of computer science at Berkeley.

A graph showing CO2 result of different AI model training processes. Picture credits: AI2

AI itself also contributes to climate change, as it takes huge computational resources to train models like GPT-3 and DALL-E. The Allen Institute for AI (AI2) conducted a study on how these training periods could be done intelligently in order to reduce their impact on the climate. It’s not a trivial calculation: where the power is coming from is constantly in flux, and peak usage, like a compute-intensive day, can’t just be split up to run the next week when the sun is out and that solar energy is abundant.

AI2’s work examines the carbon intensity of forming various patterns at various places and times, as part of a larger Green Software Foundation project to reduce the footprint of these important but energy-intensive processes.

Last but not least, OpenAI revealed this week Video pre-training (VPT), a training technique that uses a small amount of labeled data to teach an AI system to perform tasks, such as making diamond tools in Minecraft. VPT involves searching for videos on the web and having contractors produce data (e.g. 2,000 hours of videos tagged with mouse and keyboard actions) and then training a model to predict the actions based on past and future video frames. In the final step, original videos from the web are tagged with entrepreneur data to train a system to predict actions based on past footage only.

OpenAI has used Minecraft as a test case for VPT, but the company says the approach is quite general – representing a step towards “agents using general computers”. In any case, the model is available in open source, as well as the subcontracted data that OpenAI has obtained for its experiments.

Previous How to Print Background Colors or Pictures in Microsoft Word
Next Aesthetics, politics and production of Iranian visual culture in transnational circuits »