CCAC 2023 第三届智慧论辩评测(AI-Debater)



本届智慧论辩评测比赛包含2个子任务,反论点生成及基于辩题的论点生成。其中,反论点生成任务提供英文数据集,基于辩题的论点生成任务提供中文数据集。

  • 英文数据集(反论点生成任务):数据集来源于ChangeMyView论坛(CMV),标注员针对用户的交互内容进行了反驳关系标注,整理为带格式的txt文件。
  • 中文数据集(基于辩题的论点生成任务):数据集来源于2007至2021年的近700场知名华语辩论比赛,经由语音转译及人工校验得到了每场比赛的单环节、单方陈词文本,由标注员进行了论点句和互动论点对等标注,整理为带格式的txt文件。

  • 点击下载训练数据

    赛道一

    反论点生成
    介绍

    针对给定的话题和原始论点,由参赛模型自动生成反驳原始论点的5个句子(称为反论点)。


    数据样例

    • 输入:Should the phrase "under God" be retained in the Pledge of Allegiance?<tab>the under God line is actually a relatively new addition and can therefore be easily removed without significant consequences.
    • 输出:
      Just because something is new doesn't mean it lacks importance or significance, and removing it can have unforeseen consequences.
      Ignoring the historical and constitutional significance of the "under God" line in the Pledge of Allegiance is a dangerous oversimplification that fails to recognize the phrase's cultural and symbolic importance to American identity.
      ……


    评价指标

    ROUGE-L



    赛道二

    基于辩题的论点生成
    介绍

    针对既定的辩题,由参赛模型自动生成贴合辩题的5个论点。


    数据样例

    • 输入:公众事件中不应该批评不完美受害者
    • 输出:
      将矛头调转向批评不完美的受害者,使受害者与加害者之间的力量进一步失衡,不符合媒体伦理。
      舆论的变动可能影响案件的走向。
      如果秉持着应该批评不完美的心态,无疑会使得将来更少受害者敢于向公众发声。


    评价指标

    ROUGE-L



    参考文献

    1. https://eval.ai/challenge/1449/leaderboard/3606
    2. Yuan, Jian, Liying Cheng, Ruidan He, Yinzi Li, Lidong Bing, Zhongyu Wei, Qin Liu, Chenhui Shen, Shuonan Zhang, Changlong Sun, Luo Si, Changjian JIang and Xunjing Huang. Overview of Argumentative Text Understanding for AI Debater Challenge. NLPCC 2021.
    3. Lu Ji, Zhongyu Wei, Xiangkun Hu, Yang Liu, Qi Zhang and XuanJing Huang. Incorporating argument-level interactions for persuasion comments evaluation using co-attention model. COLING 2018.
    4. Lu Ji, Zhongyu Wei, Jing Li, Qi Zhang and Xuanjing Huang. Discrete Argument Representation Learning for Interactive Argument Pair Identification. NAACL 2021.
    5. Jian Yuan, Zhongyu Wei, Donghua, Zhao, Qi Zhang and Changjian Jiang. Leveraging Argumentation Knowledge Graph for Interactive Argument Pair Identification. ACL 2021 findings.
    6. Xinyu Hua, Zhe Hu, and Lu Wang. 2019. Argument Generation with Retrieval, Planning, and Realization. ACL 2019.
    7. Milad Alshomary, Shahbaz Syed, Arkajit Dhar, Martin Potthast, and Henning Wachsmuth. 2021. Counter-Argument Generation by Attacking Weak Premises. ACL 2021 findings.


             如有疑问,请致信评测会务组:disclab@fudan.edu.cn          评测官网:http://www.fudan-disc.com/sharedtask/AIDebater23/index.html