Volume 40 Issue 6
Dec.  2022
Turn off MathJax
Article Contents
Article Navigation >  Journal of Transport Information and Safety  > 2022  >  40(6): 81-91
YIN Xing, ZHANG Yu, ZHENG Qianqian, TANG Kexin. A Study of Integrated Scheduling of Automated Container Terminal Based on DDQN[J]. Journal of Transport Information and Safety, 2022, 40(6): 81-91. doi: 10.3963/j.jssn.1674-4861.2022.06.009
Citation: YIN Xing, ZHANG Yu, ZHENG Qianqian, TANG Kexin. A Study of Integrated Scheduling of Automated Container Terminal Based on DDQN[J]. Journal of Transport Information and Safety, 2022, 40(6): 81-91. doi: 10.3963/j.jssn.1674-4861.2022.06.009
shu

A Study of Integrated Scheduling of Automated Container Terminal Based on DDQN

doi: 10.3963/j.jssn.1674-4861.2022.06.009
  • 1.

    School of Transportation and Logistics Engineering, Wuhan University of Technology, Wuhan 430063, China

  • 2.

    Inland Port and Shipping Industry Research Co. Ltd. of Guangdong Province, Shaoguan 512000, Guangdong, China

  • Received Date: 2022-06-13
    Available Online: 2023-03-27
    Abstract
  • The interactive operations of quay cranes, artificial intelligent robots of transportation(ARTs), and yard cranes during automatic container terminal unloading are studied. A three-stages integrated scheduling model of automated container terminal based on hybrid flow shop scheduling problem is proposed, with the criterion of minimizing the makespan. In addition, the scheduling environment requires high real-time response. A deep reinforcement learning algorithm, namely double deep Q-network(DDQN), is used to solve the problem of dynamic characteristics of the automatic terminal scheduling environment. The input of the model is the real-time status data of the equipment at each stage. The neural network is used to fit the value-action function. The model is trained by experience playback mechanism. The single heuristic rule with the compound heuristic rule is taken as the equipment candidate behavior. By strengthening the learning action selection and action evaluation mechanism, the optimal container equipment combination strategy is obtained. According to the actual survey data of Tianjin Port Automation Terminal, different scales cases are designed for experimental comparison and analysis. The results show that: the total operation time of the proposed method is reduced by 7.84% on average compared with the current advanced particle swarm optimization algorithm, and the gap with the theoretical lower bound value is 6.0%, 5.6%, and 4.6%, respectively. In addition, the equipment loading in the three stages is relatively balanced. And the average utilization rate of equipment is 89%, which can meet the actual application requirements. In small-scale examples, the average error of the total completion time obtained by DDQN is 1.99% compared with Gurobi. With the increase of the size of the example, the solving time is increased by 59% at most, which verifies the feasibility and efficiency of the proposed method for improving the operation efficiency of the automated container terminal.

     

    • FullText(HTML)
  • loading
    • References(25)
  • [1]
    高雪峰. 基于深度强化学习的自动化集装箱码头双场桥动态调度研究[D]. 大连: 大连理工大学, 2020.

    GAO X F. Research on dynamic scheduling of two YC in automated container terminal based on deep reinforcement learning[D]. Dalian: Dalian University of Technology, 2020. (in Chinese)
    [2]
    丁一, 袁浩, 方怀瑾, 等. 考虑冲突规避的自动化集装箱码头AGV优化调度方法[J]. 交通信息与安全, 2022, 40(3): 96-107. doi: 10.3963/j.jssn.1674-4861.2022.03.010

    DING Y, YUAN H, FANG H J, et al. An optimal scheduling method of AGVs at automated container terminal considering conflict avoidance[J]. Journal of Transport Information and Safety, 2022, 40(3): 96-107. (in Chinese) doi: 10.3963/j.jssn.1674-4861.2022.03.010
    [3]
    夏孟珏, 史学鑫, 李美贞. 集装箱码头岸桥突发故障情况下装卸船作业重调度研究[J]. 上海海事大学学报, 2022, 43(1): 30-37. https://www.cnki.com.cn/Article/CJFDTOTAL-SHHY202201005.htm

    XIA M J, SHI X X, LI M Z, et al. Study on handling operation rescheduling under sudden malfunction of container terminal quay cranes[J]. Journal of Shanghai Maritime University, 2022, 43(1): 30-37. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-SHHY202201005.htm
    [4]
    梁承姬, 杨全业. 双循环操作策略下集装箱码头岸桥与集卡多船作业联合调度[J]. 重庆交通大学学报(自然科学版), 2018, 37(3): 106-114. https://www.cnki.com.cn/Article/CJFDTOTAL-CQJT201803019.htm

    LIANG C J, YANG Q Y. Container terminal QC & IT integral scheduling model for multi-vessel operation under double-cycle strategy[J]. Journal of Chongqing Jiaotong University(Natural Science), 2018, 37(3): 106-114. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-CQJT201803019.htm
    [5]
    陈超, 邱建梅, 台伟力, 等. 出口箱随机入港下的集装箱码头泊位调度[J]. 交通信息与安全, 2014, 181(1): 91-96. doi: 10.3963/j.issn.1674-4861.2014.01.019

    CHEN C, QIU J M, TAI W L, et al. Berth allocation planning in container terminals for the outbound containers arrival in random order[J]. Journal of Transport Information and Safety, 2014, 181(1): 91-96. (in Chinese) doi: 10.3963/j.issn.1674-4861.2014.01.019
    [6]
    常祎妹, 朱晓宁. 不确定因素下的集装箱码头车船间装卸作业集成调度[J]. 交通运输工程学报, 2017, 17(6): 115-124. https://www.cnki.com.cn/Article/CJFDTOTAL-JYGC201706017.htm

    CHANG Y M, ZHU X N. Integrated scheduling of handling operation between train and vessel in container terminal under uncertain factor[J]. Journal of Traffic and Transportation Engineering, 2017, 17(6): 115-124. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JYGC201706017.htm
    [7]
    KIM K H, PARK Y M. A crane scheduling method for port container terminals[J]. European Journal of Operational Research, 2004, 156(3): 752-768.
    [8]
    秦天保, 葛浩, 沙梅. 约束规划求解集装箱装卸系统集成调度问题[J]. 系统工程理论与实践, 2015, 35(8): 2127-2136. https://www.cnki.com.cn/Article/CJFDTOTAL-XTLL201508023.htm

    QIN T B, GE H, SHA M. Constraint programming for the integrated scheduling problem of container handling system in container terminals[J]. System Engineering-Theory & Practice, 2015, 35(8): 2127-2136. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-XTLL201508023.htm
    [9]
    ZHEN L, YU S C, WANG S A, et al. Scheduling quay cranes and yard trucks for unloading operations in container ports[J]. Annals of Operations Research, 2019, 273(1): 455-478.
    [10]
    钟祾充, 李文锋, 贺利军, 等. 集装箱码头混合零空闲柔性流水作业调度优化[J/OL]. 计算机集成制造系统: 1-22[2022-10-28]. http://kns.cnki.net/kcms/detail/11.5946.TP.20211228.1358.016.html.

    ZHONG L C, LI W F, HE L J, et al. Optimization of mixed no-idle flexible flow scheduling in container terminal[J]. Computer Integrated Manufacturing Systems: 1-22[2022-10-28]. http://kns.cnki.net/kcms/detail/11.5946.TP.20211228.1358.016.html. (in Chinese)
    [11]
    陈超, 张哲, 曾庆成. 集装箱码头混合交叉作业集成调度模型[J]. 交通运输工程学报, 2012, 12(3): 92-100. https://www.cnki.com.cn/Article/CJFDTOTAL-JYGC201203017.htm

    CHEN C, ZHANG Z, ZENG Q C. Integrated scheduling model of mixed cross-operation for container terminal[J]. Journal of Traffic and Transportation Engineering, 2012, 12(3): 92-100. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JYGC201203017.htm
    [12]
    杨彩云, 张煜, 徐亚军, 等. 自动化集装箱码头ART动态调速群智决策研究[J]. 武汉理工大学学报, 2022, 44(1): 28-35. https://www.cnki.com.cn/Article/CJFDTOTAL-WHGY202201005.htm

    YANG C Y, ZHANG Y, XU Y J, et al. Simulation analysis of ART dynamic speed regulation of automated container terminal based on MAS[J]. Journal of Wuhan University of Technology, 2022, 44(1): 28-35. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-WHGY202201005.htm
    [13]
    SUTTON R S, BARTO A G. Reinforcement learning: an introduction[J]. IEEE Transactions on Neural Networks, 1998, 9(5): 1054-1054.
    [14]
    张华胜. 基于深度强化学习的自动化码头跨运车集成调度研究[D]. 上海: 上海海事大学, 2021.

    ZHANG H S. Research on integrated scheduling of shuttle carriers in automated terminals based on deep reinforcement learning[D]. Shanghai: Shanghai Maritime University, 2021. (in Chinese)
    [15]
    尚晶, 徐长生. 基于强化学习的集装箱码头卡车调度策略研究[J]. 武汉理工大学学报, 2011, 33(3): 72-76. https://www.cnki.com.cn/Article/CJFDTOTAL-WHGY201103015.htm

    SHANG J, XU C S. Vehicle scheduling in container terminal based on reinforcement learning[J]. Journal of Wuhan University of Technology, 2011, 33(3): 72-76. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-WHGY201103015.htm
    [16]
    PARK S J, YOO Y H, PYO C W. Applying DQN solutions in fog-based vehicular networks: Scheduling, caching, and collision control[J]. Vehicular Communications, 2022, 33(C): 100397.
    [17]
    SALVADORl M S. A solution to a special class of flow shop scheduling problems[J]. Symposium on the Theory of Scheduling and Its Applications, 1972(86): 83-91.
    [18]
    SUTTON R S, BARTO A G. Reinforcement Learning: An Introduction[M]. second edition. 北京: 电子工业出版, 2019.
    [19]
    陆志强, 任逸飞, 许则鑫. 基于深度学习的资源投入问题算法[J]. 计算机集成制造系统, 2021, 27(6): 1558-1568. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJJ202106003.htm

    LU Z Q, REN Y F, XU Z X. Research on the deep learning algorithm for resource investment problem[J]. Computer Integrated Manufacturing Systems, 2021, 27(6): 1558-1568. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JSJJ202106003.htm
    [20]
    HAN B A, YANG J J. Research on adaptive job shop scheduling problems based on dueling double DQN[J]. IEEE Access, 2020(8): 186474-186495.
    [21]
    WANG Y H, ZHOU X, SHI Y X, et al. Transmission network expansion planning considering wind power and load uncertainties based on multi-agent DDQN[J]. Energies, 2021, 14(19): 6073.
    [22]
    LIU C L, CHANG C C, TSENG C J. Actor-Critic deep reinforcement learning for solving job shop scheduling problem[J]. IEEE Access, 2020(8): 71752-71762.
    [23]
    邢曦文. 基于混合流水作业组织的集装箱码头装卸集成调度优化[D]. 大连: 大连海事大学, 2013.

    XING X W. Optimization of container loading/unloading integrated scheduling in a container terminal based on hybrid flowshop[D]. Dalian: Dalian Maritime University, 2013. (in Chinese)
    [24]
    肖鹏飞, 张超勇, 孟磊磊, 等. 基于深度强化学习的非置换流水车间调度问题[J]. 计算机集成制造系统, 2021, 27(1): 192-205. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJJ202101018.htm

    XIAO P F, ZHANG C Y, MENG L L, et al. Non-permutation flow shop scheduling problem based on deep reinforcement learning[J]. Computer Integrated Manufacturing Systems, 2021, 27(1): 192-205. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JSJJ202101018.htm
    [25]
    YUTTHAPONG T, WERASAK K. Comparing nonlinear inertia weights and constriction factors in particle swarm optimization[J]. International Journal of Knowledge-Based and Intelligent Engineering Systems, 2011, 15(2): 65-70.
    • Relative Articles
    • Supplements(0)
    • Cited By
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(8)  / Tables(10)

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (1137) PDF downloads(56) Cited by()
    Proportional views
    Related

    Copyright Editorial Office of Transport Information and Safety鄂ICP备05003336号-2

    Address：No. 1178，Heping Avenue, Wuchang, Wuhan, CHINA (430063)

    Tel：027-86580355 Email：jtjsj@vip.163.com

    Supported by: Beijing Renhe Information Technology Co. Ltd  

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return