Revisiting Neural Networks for Continual Learning: An Architectural Perspective
Abstract
Normative Restraining Bolts (NRBs) adapt the restraining bolt technique (originally developed for safe reinforcement learning) to ensure compliance with social, legal, and ethical norms. While effective, NRBs rely on trial-and-error weight tuning, which hinders their ability to enforce hierarchical norms; moreover, norm updates require retraining. In this paper, we reformulate learning with NRBs as a multi-objective reinforcement learning (MORL) problem, where each norm is treated as a distinct objective. This enables the introduction of Ordered Normative Restraining Bolts (ONRBs), which support algorithmic weight selection, prioritized norms, norm updates, and provide formal guarantees on minimizing norm violations. Case studies show that ONRBs offer a robust and principled foundation for RL-agents to comply with a wide range of norms while achieving their goals.
Cite
Text
Lu et al. "Revisiting Neural Networks for Continual Learning: An Architectural Perspective." International Joint Conference on Artificial Intelligence, 2024. doi:10.24963/ijcai.2024/514Markdown
[Lu et al. "Revisiting Neural Networks for Continual Learning: An Architectural Perspective." International Joint Conference on Artificial Intelligence, 2024.](https://mlanthology.org/ijcai/2024/lu2024ijcai-revisiting/) doi:10.24963/ijcai.2024/514BibTeX
@inproceedings{lu2024ijcai-revisiting,
title = {{Revisiting Neural Networks for Continual Learning: An Architectural Perspective}},
author = {Lu, Aojun and Feng, Tao and Yuan, Hangjie and Song, Xiaotian and Sun, Yanan},
booktitle = {International Joint Conference on Artificial Intelligence},
year = {2024},
pages = {4651-4659},
doi = {10.24963/ijcai.2024/514},
url = {https://mlanthology.org/ijcai/2024/lu2024ijcai-revisiting/}
}