Anno: 2022

openreview_logo

Learning to Explore Multiple Environments without Rewards

Authors Mirco Mutti, Mattia Mancassola, Marcello Restelli Abstract Several recent works have been dedicated to the pure exploration of a single reward-free environment. Along this line, we address the problem of learning to explore a class of multiple reward-free environments with a unique general strategy, which aims to provide a universal initialization to subsequent reinforcement […]
Read More
openreview_logo

Meta Learning the Step Size in Policy Gradient Methods

Authors Luca Sabbioni, Francesco Corda, Marcello Restelli Abstract Policy-based algorithms are among the most widely adopted techniques in model-free RL, thanks to their strong theoretical groundings and good properties in continuous action spaces. Unfortunately, these methods require precise and problem-specific hyperparameter tuning to achieve good performance and, as a consequence, they tend to struggle when […]
Read More
PMLR

Leveraging Good Representations in Linear Contextual Bandits

Authors Matteo Papini, Andrea Tirinzoni, Marcello Restelli, Alessandro Lazaric, Matteo Pirotta Abstract The linear contextual bandit literature is mostly focused on the design of efficient learning algorithms for a given representation. However, a contextual bandit problem may admit multiple linear representations, each one with different characteristics that directly impact the regret of the learning algorithm. In particular, recent works […]
Read More
PMLR

Provably Efficient Learning of Transferable Rewards

Authors Alberto Maria Metelli, Giorgia Ramponi, Alessandro Concetti, Marcello Restelli  Abstract The reward function is widely accepted as a succinct, robust, and transferable representation of a task. Typical approaches, at the basis of Inverse Reinforcement Learning (IRL), leverage on expert demonstrations to recover a reward function. In this paper, we study the theoretical properties of the class of […]
Read More
Nature logo

Quantum compiling by deep reinforcement learning

Authors Lorenzo Moro, Matteo G. A. Paris, Marcello Restelli, Enrico Prati Abstract The general problem of quantum compiling is to approximate any unitary transformation that describes the quantum computation as a sequence of elements selected from a finite base of universal quantum gates. The Solovay-Kitaev theorem guarantees the existence of such an approximating sequence. Though, […]
Read More
Machine-Learning

Dealing with multiple experts and non-stationarity in inverse reinforcement learning: an application to real-life problems

Authors Amarildo Likmeta, Alberto Maria Metelli, Giorgia Ramponi, Andrea Tirinzoni, Matteo Giuliani, Marcello Restelli Abstract In real-world applications, inferring the intentions of expert agents (e.g., human operators) can be fundamental to understand how possibly conflicting objectives are managed, helping to interpret the demonstrated behavior. In this paper, we discuss how inverse reinforcement learning (IRL) can […]
Read More
Upstream-oil-and-gas-technology

Data-driven indicators for the detection and prediction of stuck-pipe events in oil&gas drilling operations

Abstract Stuck-pipe phenomena can have disastrous effects on drilling performance, with outcomes that can range from time delays to loss of expensive machinery. In this work, we develop three indicators based on mudlog data, which aim to detect three different physical phenomena associated with theinsurgence of a sticking. In particular, two indices target respectively the detection […]
Read More
Machine-Learning

Policy space identification in configurable environments

Authors Alberto Maria Metelli, Guglielmo Manneschi, Marcello Restelli Abstract We study the problem of identifying the policy space available to an agent in a learning process, having access to a set of demonstrations generated by the agent playing the optimal policy in the considered space. We introduce an approach based on frequentist statistical testing to […]
Read More
Machine-Learning-and-Knowledge-Discovery-in-Database

Exploiting History Data for Nonstationary Multi-armed Bandit

Authors Gerlando Re, Fabio Chiusano, Francesco Trovò, Diego Carrera, Giacomo Boracchi, Marcello Restelli. Abstract The Multi-armed Bandit (MAB) framework has been applied successfully in many application fields. In the last years, the use of active approaches to tackle the nonstationary MAB setting, i.e., algorithms capable of detecting changes in the environment and re-configuring automatically to […]
Read More
Machine-Learning-and-Knowledge-Discovery-in-Database

Conservative Online Convex Optimization

Authors Martino Bernasconi de Luca, Edoardo Vittori, Francesco Trovò, Marcello Restelli Abstract Online learning algorithms often have the issue of exhibiting poor performance during the initial stages of the optimization procedure, which in practical applications might dissuade potential users from deploying such solutions. In this paper, we study a novel setting, namely conservative online convex […]
Read More