Site icon News Azi

Mimicking human sleep as a way to prevent catastrophic forgetting in AI systems

Representation of memories in an abstract synaptic space and its evolution with and without sleep. Credit: Golden R, Delanois JE, Sanda P, and Bazhenov M, 2022, PLOS Computational Biology, CC-BY 4.0 (

A trio of researchers from the University of California, working with a colleague from the Institute of Computer Science of the Czech Academy of Sciences, has found that it is possible to prevent catastrophic forgetting in AI systems by having such systems mimic human REM sleep.

In their paper published in PLOS Computational Biology, Ryan Golden, Jean Erik Delanois, Maxim Bazhenov and Pavel Sanda describe teaching artificial intelligence systems to remember what was learned from a beginning task when working on a second task.

Prior research has shown that people experience something called consolidation of memory during REM sleep. It is a process whereby things that were experienced recently are moved to long term memory to make room for new experiences. Without such a process, the brain undergoes catastrophic forgetting, where memories of recent things are not retained.

This can be seen in some older people who lose the ability to sleep well, and thus find themselves able to remember things from the distant past but not things that happened in recent days. In this new effort, the researchers have found that something similar can be used to help AI systems keep what has been learned in the past while learning about new things.

AI systems have become well-known for their ability to master a certain genre—one can be created to become a chess master, for example. But getting AI systems to master more than one topic has proved to be challenging.

This is because, the researchers explain, new learning tends to come at the expense of old learning. As more of something in a new area is learned, more of the old memories are lost until they disappear altogether. To overcome this problem, the researchers looked to how the human brain handles similar situations.

First, they built an AI system that was taught first one task, then another. They found, as expected, that as it became better at the second task it lost abilities in the first task. To overcome the problem the researchers added code that mimicked REM sleep in the human brain. They essentially gave the system the ability to intersperse sleep/work phases that allowed the system to continue to hold on to older memories as new ones were being processed and that helped to prevent catastrophic forgetting.

More information:
Sleep prevents catastrophic forgetting in spiking neural networks by forming a joint synaptic weight representation, PLOS Computational Biology (2022). … journal.pcbi.1010628

© 2022 Science X Network

Mimicking human sleep as a way to prevent catastrophic forgetting in AI systems (2022, November 24)
retrieved 24 November 2022

This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.

Stay connected with us on social media platform for instant update click here to join our  Twitter, & Facebook

We are now on Telegram. Click here to join our channel (@TechiUpdate) and stay updated with the latest Technology headlines.

For all the latest Technology News Click Here 

 For the latest news and updates, follow us on Google News

Read original article here

Denial of responsibility! NewsAzi is an automatic aggregator around the global media. All the content are available free on Internet. We have just arranged it in one platform for educational purpose only. In each content, the hyperlink to the primary source is specified. All trademarks belong to their rightful owners, all materials to their authors. If you are the owner of the content and do not want us to publish your materials on our website, please contact us by email – The content will be deleted within 24 hours.
Exit mobile version