EFECTIW-ROTER: Deep Reinforcement Learning Approach for Solving Heterogeneous Fleet and Demand Vehicle Routing Problem With Time-Window Constraints

EasyChair Preprint 14918

Abstract

Logistics operations are significantly burdened by distribution costs, which constitute nearly half of the total expenses. Central to reducing these costs is optimizing delivery routes by addressing the heterogeneous fleet and demand vehicle routing problem with time-window constraints (HFDVRPTW). In this paper, we propose a deep reinforcement learning (DRL)-based method, termed spatial Edge-Feature EnhanCed mulTI-graph fusion encoder With spectral-based embedding and hieRarchical decOder with learnable TEmpoRal positional embedding (EFECTIW-ROTER, pronounced "Effective Router"), to tackle this practical and complex optimization problem. EFECTIW-ROTER utilizes two sparse graphs to represent node connectivity, where nodes correspond to customers and the depot. This sparsity results from the time-window constraints and customers' demand relative to the list of acceptable vehicle attributes specified for service within a heterogeneous fleet, determined by the reachability of the nodes based on these two factors. Leveraging two graph Transformer models, EFECTIW-ROTER's encoding module effectively captures the interactions between the nodes based on these factors. One model encodes customers' heterogeneous demand with spatial edge features based on travel time between the nodes, while the second employs temporal positional embeddings to capture temporal relationships based on time-window ordering. A fusion model is introduced to integrate node interactions based on these graphs. Additionally, a spectral-attention-based pooling ensures effective state representation for the DRL-based method. EFECTIW-ROTER features a hierarchical attention decoder operating in two stages: heterogeneous vehicle selection and node selection. Enhanced with positional embeddings, the decoder is empowered to make effective routing decisions based on time-window constraints' ordering.

Keyphrases: Attention Model, Reinforcement Learning, Spatial-temporal systems, combinatorial optimization

@booklet{EasyChair:14918,
  author    = {Arash Mozhdehi and Mahdi Mohammadizadeh and Yunli Wang and Sun Sun and Xin Wang},
  title     = {EFECTIW-ROTER: Deep Reinforcement Learning Approach for Solving Heterogeneous Fleet and Demand Vehicle Routing Problem With Time-Window Constraints},
  howpublished = {EasyChair Preprint 14918},
  year      = {EasyChair, 2024}}