Anno: 2022

Authors Alberto Maria Metelli, Samuele Meta, Marcello Restelli Abstract Off-policy methods are the basis of a large number of effective Policy Optimization (PO) algorithms. In this setting, Importance Sampling (IS) is typically employed as a what-if analysis tool, with the goal of estimating the performance of a target policy, given samples collected with a different […]

Authors Lorenzo Moro, Amarildo Likmeta, Enrico Prati, Marcello Restelli Abstract We consider the problem of goal-directed planning under a deterministic transition model. Monte Carlo Tree Search has shown remarkable performance in solving deterministic control problems. It has been extended from complex continuous domains through function approximators to bias the search of the planning tree in […]

Authors Alberto Maria Metelli, Samuele Meta, Marcello Restelli Abstract Off-policy methods are the basis of a large number of effective Policy Optimization (PO) algorithms. In this setting, Importance Sampling (IS) is typically employed as a what-if analysis tool, with the goal of estimating the performance of a target policy, given samples collected with a different […]

Authors Giuseppe Canonaco, Andrea Soprani, Matteo Giuliani, Andrea Castelletti, Manuel Roveri, Marcello Restelli Abstract In most of the transfer learning approaches to reinforcement learning (RL) the distribution over the tasks is assumed to be stationary. Therefore, the target and source tasks are i.i.d. samples of the same distribution. Unfortunately, this assumption rarely holds in real-world […]

Authors Matteo Papini, Andrea Tirinzoni, Aldo Pacchiano, Marcello Restelli, Alessandro Lazaric, Matteo Pirotta Abstract We study the role of the representation of state-action value functions in regret minimization in finite-horizon Markov Decision Processes (MDPs) with linear structure. We first derive a necessary condition on the representation, called universally spanning optimal features (UNISOFT), to achieve constant […]

Authors Giorgia Ramponi, Alberto Maria Metelli, Alessandro Concetti, Marcello Restelli Abstract The Configurable Markov Decision Process framework includes two entities: a Reinforcement Learning agent and a configurator that can modify some environmental parameters to improve the agent’s performance. This presupposes that the two actors have the same reward functions. What if the configurator does not […]

Authors Alberto Maria Metelli, Alessio Russo, Marcello Restelli Abstract Importance Sampling (IS) is a widely used building block for a large variety of off-policy estimation and learning algorithms. However, empirical and theoretical studies have progressively shown that vanilla IS leads to poor estimations whenever the behavioral and target policies are too dissimilar. In this paper, […]

Authors Marco Mussi, Luigi Pellegrino, Marcello Restelli, Francesco Trovò. Abstract In recent years, the use of Lithium-ion batteries in smart power systems and hybrid/electric vehicles has become increasingly popular since they provide a flexible and cost-effective way to store and deliver power. Their full integration into more complex systems requires an accurate estimate of the […]

Authors Alberto Metelli, Andrea Castelletti, Marcello Restelli. Abstract A drought is a slowly developing natural phenomenon that can occur in all climatic zones and can be defined as a temporary but significant decrease in water availability. Over the past three decades, the cost of droughts in Europe has amounted to over 100 billion euros and […]