Scaling decision theoretic planning to large multiagent systems is challenging dueto uncertainty and partial observability in the environment. We focus on a multia-gent planning model subclass, relevant to urban settings, where agent interactionsare dependent on their “collective influence” on each other, rather than their iden-tities. Unlike previous work, we address a general setting where system rewardisnot decomposableamong agents. We develop collective actor-critic RL ap-proaches for this setting, and address the problem of multiagent credit assignment,and computing low variance policy gradient estimates that result in faster conver-gence to high quality solutions. We also developdifference rewardsbased creditassignment methods for the collective setting. Empirically our new approachesprovide significantly better solutions than previous methods in the presence ofglobal rewards on two real world problems modeling taxi fleet optimization andmultiagent patrolling, and a synthetic grid navigation domain.

Effective traffic management can help port operators gain a competitive edge in service level and efficient use of limited resources. One critical aspect of traffic management is gate operations management, ensuring a good customer experience to logistic carriers and considering the impact of congestion in and around the port. In this paper, we describe the design and implementation of a decision support tool to help gate operators plan for future scenarios with fluctuating demand and limited resources. We propose a simulationoptimization framework which incorporates theoretical results from queuing theory to approximate complex multi-lane multi-server systems. Our major contribution in this paper is the demonstration that the proposed design, when coupled with real data, can indeed help port operators improve their performances. To provide concrete real-world evidence that such technology has benefits, we have tested the system operationally since December 2017 and present the results and analysis in this paper.

Traditional taxi fleet operators world-over have been facing intense competitions from various ride-hailing services such as Uber and Grab. Based on our studies on the taxi industry in Singapore, we see that the emergence of Uber and Grab in the ride-hailing market has greatly impacted the taxi industry: the average daily taxi ridership for the past two years has been falling continuously, by close to 20% in total. In this work, we discuss how efficient real-time data analytics and large-scale multiagent optimization technology could help taxi drivers compete against more technologically advanced service platforms. Our system has been in field trial with close to 400 drivers, and our initial results show that by following our recommendations, drivers on average save 21.5% on roaming time.

Traditional taxi fleet operators world-over have been facing intense competitions from various ride-hailing services such as Uber and Grab (specific to the Southeast Asia region). Based on our studies on the taxi industry in Singapore, we see that the emergence of Uber and Grab in the ride-hailing market has greatly impacted the taxi industry: the average daily taxi ridership for the past two years has been falling continuously, by close to 20% in total. In this work, we discuss how efficient real-time data analytics and large-scale multi-agent optimization technology could potentially help taxi drivers compete against more technologically advanced service platforms.

In most cities, taxis play an important role in providing point-to-point transportation service. If the taxi service is reliable,responsive, and cost-effective, past studies show that taxi-like services can be a viable choice in replacing a significant amount of private cars. However, making taxi services efficient is extremely challenging, mainly due to the fact that taxi drivers are self-interested and they operate with only local information. Although past research has demonstrated how recommendation systems could potentially help taxi drivers in improving their performance, most of these efforts are not feasible in practice. This is mostly due to the lack of both the comprehensive data coverage and an efficient recommendation engine that can scale to tens of thousands of drivers.

We address the problem of mitigating congestion and preventing hotspots in busy water areas such as Singapore Straits and port waters. Increasing maritime traffic coupled with narrow waterways makes vessel schedule coordination for just-in-time arrival critical for navigational safety. Our contributions are: 1) We formulate the maritime traffic management problem based on the real case study of Singapore waters; 2) We model the problem as a variant of the resourceconstrained project scheduling problem (RCPSP), and formulate mixed-integer and constraint programming (MIP/CP) formulations; 3) To improve the scalability, we develop a combinatorial Benders (CB) approach that is significantly more effective than standard MIP and CP formulations. We also develop symmetry breaking constraints and optimality cuts that further enhance the CB approach’s effectiveness; 4) We develop a realistic maritime traffic simulator using electronic navigation charts of Singapore Straits. Our scheduling approach on synthetic problems and a real 55-day AIS dataset results in significant reduction of the traffic density while incurring minimal delays.

The Vehicle Routing Problem with Location Congestion (VRPLC) integrates Vehicle Routing Problem (VRP) and the location congestion constraints (e.g., due to docking capacity). The capacity of any location that needs to be served by multiple vehicles is assumed to be limited, so that only a limited number of vehicles can visit a location in any particular time period. The goal for VRPLC is to minimize the weighted summation of vehicle travel time between visited locations and vehicle waiting time induced by the visited locations congestion. We formulate the problem into a Mixed Integer Programming (MIP) model and propose an exact approach using Benders decomposition to solve it.

Mobile crowdsourcing, which involves the use of a pool of crowd-workers who visit different locations to perform a variety of location-specific tasks, has emerged as a key paradigm for executing many urban services. While on-demand crowdsourced transportation (e.g. Uber) has arguably received the greatest attention, last mile logistics and package delivery is another service that is rapidly adopting this crowdsourcing paradigm. Crowdsourced package delivery has several key advantages: (1) Logistics companies no longer have to maintain a large dedicated fleet and worker pool, thus reducing their capital expenses; (2) They can tap on a flexible workforce that can handle seasonal demand fluctuations. Despite avid interests in adopting the crowdsourcing paradigm, most current practices suffer from the inefficiency caused by decentralized task allocation. This is so since crowd-workers need to independently browse and choose their preferred tasks, which in most cases are cognitively demanding and rarely optimal.