最常见NLP任务练手项目汇总

深碍√TFBOYSˉ_ 2022-02-01 14:57 348阅读 0赞

1.  分词 Word Segmentation  
    [chqiwang/convseg][chqiwang_convseg] ，基于CNN做中文分词，提供数据和代码。
2.  词预测 Word Prediction  
    [Kyubyong/word\_prediction][Kyubyong_word_prediction] ，基于CNN做词预测，提供数据和代码。
3.  文本蕴涵 Textual Entailment  
    [Steven-Hewitt/Entailment-with-Tensorflow][Steven-Hewitt_Entailment-with-Tensorflow]，基于Tensorflow做文本蕴涵，提供数据和代码。
4.  语音识别 Automatic Speech Recognition  
    [buriburisuri/speech-to-text-wavenet][buriburisuri_speech-to-text-wavenet]，基于DeepMind WaveNet和Tensorflow做句子级语音识别。
5.  自动摘要 Automatic Summarisation  
    [PKULCWM/PKUSUMSUM][PKULCWM_PKUSUMSUM]，北大万小军老师团队的自动摘要方法汇总，包含了他们大量paper的实现，支持单文档摘要、多文档摘要、topic-focused多文档摘要。
6.  文本纠错 Text Correct  
    [atpaino/deep-text-corrector][atpaino_deep-text-corrector]，基于深度学习做文本纠错，提供数据和代码。
7.  字音转换 Grapheme to Phoneme  
    [cmusphinx/g2p-seq2seq][cmusphinx_g2p-seq2seq]，基于网红transformer做， 提供数据和代码。
8.  复述检测 Paraphrase Detection 和 问答 Question Answering  
    Paraphrase-Driven Learning for Open Question Answering， 基于复述驱动学习的开放域问答。
9.  音汉互译 Pinyin-To-Chinese  
    [Kyubyong/neural\_chinese\_transliterator][Kyubyong_neural_chinese_transliterator]，基于CNN做音汉互译。
10. 情感分析 Sentiment Analysis  
    情感分析保护的内容太多了，目前没发现比较全的。推荐两个适合练手的吧：Deeply Moving: Deep Learning for Sentiment Analysis，[http://sentic.net/about/。][http_sentic.net_about]
11. 手语识别 [Sign Language Recognition][]
12. 该项目在手语识别做的非常成熟。
13. 词性标注（POS）、 命名实体识别（NER）、 句法分析（parser）、 语义角色标注（SRL） 等。  
    [HIT-SCIR/ltp][HIT-SCIR_ltp]， 保护代码、模型、数据，还有详细的文档，而且效果还很好。
14. 词干 Word Stemming  
    [snowballstem/snowball][snowballstem_snowball]， 实现的词干效果还不错。
15. 语言识别 Language Identification  
    [https://github.com/saffsd/langid.py，语言识别比较好的开源工具。][https_github.com_saffsd_langid.py]
16. 机器翻译 Machine Translation  
    [OpenNMT/OpenNMT-py][OpenNMT_OpenNMT-py]， 基于PyTorch的神经机器翻译，很适合练手。
17. 复述生成 Paraphrase Generation  
    vsuthichai/paraphraser，基于Tensorflow的句子级复述生成，适合练手。
18. 关系抽取 Relationship Extraction  
    ankitp94/relationship-extraction，基于核方法的关系抽取。
19. 句子边界消歧 Sentence Boundary Disambiguation  
    [https://github.com/Orekhov/SentenceBreaking，很有意思。][https_github.com_Orekhov_SentenceBreaking]

19.事件抽取 Event Extraction  
[liuhuanyong/ComplexEventExtraction][liuhuanyong_ComplexEventExtraction]， 中文复合事件抽取，包括条件事件、因果事件、顺承事件、反转事件等事件抽取，并形成事理图谱。

1.  词义消歧 Word Sense Disambiguation  
    [alvations/pywsd][alvations_pywsd]，代码不多，方法简单，适合练手。
2.  命名实体消歧 Named Entity Disambiguation  
    [dice-group/AGDISTIS][dice-group_AGDISTIS]，实体消歧是很重要的，尤其对于实体融合（比如知识图谱中多源数据融合）、实体链接。
3.  幽默检测 Humor Detection  
    [pln-fing-udelar/pghumor][pln-fing-udelar_pghumor]
4.  讽刺检测 Sarcasm Detection  
    [AniSkywalker/SarcasmDetection][AniSkywalker_SarcasmDetection]，基于神经网络的讽刺检测。
5.  实体链接 Entity Linking  
    [hasibi/EntityLinkingRetrieval-ELR][hasibi_EntityLinkingRetrieval-ELR]， 实体链接用途非常广，非常适合练手。
6.  指代消歧 Coreference Resolution  
    [huggingface/neuralcoref][huggingface_neuralcoref]，基于神经网络的指代消歧。
7.  关键词/短语抽取和社会标签推荐 Keyphrase Extraction and Social Tag Suggestion  
    [thunlp/THUTag][thunlp_THUTag], 用多种方法 实现了多种关键词/短语抽取和社会标签推荐。

[chqiwang_convseg]: https://github.com/chqiwang/convseg
[Kyubyong_word_prediction]: https://github.com/Kyubyong/word_prediction
[Steven-Hewitt_Entailment-with-Tensorflow]: https://github.com/Steven-Hewitt/Entailment-with-Tensorflow
[buriburisuri_speech-to-text-wavenet]: https://github.com/buriburisuri/speech-to-text-wavenet
[PKULCWM_PKUSUMSUM]: https://github.com/PKULCWM/PKUSUMSUM
[atpaino_deep-text-corrector]: https://github.com/atpaino/deep-text-corrector
[cmusphinx_g2p-seq2seq]: https://github.com/cmusphinx/g2p-seq2seq
[Kyubyong_neural_chinese_transliterator]: https://github.com/Kyubyong/neural_chinese_transliterator
[http_sentic.net_about]: http://sentic.net/about/%E3%80%82
[Sign Language Recognition]: https://github.com/hthuwal/sign-language-gesture-recognition
[HIT-SCIR_ltp]: https://github.com/HIT-SCIR/ltp
[snowballstem_snowball]: https://github.com/snowballstem/snowball
[https_github.com_saffsd_langid.py]: https://github.com/saffsd/langid.py%EF%BC%8C%E8%AF%AD%E8%A8%80%E8%AF%86%E5%88%AB%E6%AF%94%E8%BE%83%E5%A5%BD%E7%9A%84%E5%BC%80%E6%BA%90%E5%B7%A5%E5%85%B7%E3%80%82
[OpenNMT_OpenNMT-py]: https://github.com/OpenNMT/OpenNMT-py
[https_github.com_Orekhov_SentenceBreaking]: https://github.com/Orekhov/SentenceBreaking%EF%BC%8C%E5%BE%88%E6%9C%89%E6%84%8F%E6%80%9D%E3%80%82
[liuhuanyong_ComplexEventExtraction]: https://github.com/liuhuanyong/ComplexEventExtraction
[alvations_pywsd]: https://github.com/alvations/pywsd
[dice-group_AGDISTIS]: https://github.com/dice-group/AGDISTIS
[pln-fing-udelar_pghumor]: https://github.com/pln-fing-udelar/pghumor
[AniSkywalker_SarcasmDetection]: https://github.com/AniSkywalker/SarcasmDetection
[hasibi_EntityLinkingRetrieval-ELR]: https://github.com/hasibi/EntityLinkingRetrieval-ELR
[huggingface_neuralcoref]: https://github.com/huggingface/neuralcoref
[thunlp_THUTag]: https://github.com/thunlp/THUTag