lccc

참고자료:

크기가 큰

TFDS에 이 데이터세트를 로드하려면 다음 명령어를 사용하세요.

ds = tfds.load('huggingface:lccc/large')
  • 설명 :
LCCC: Large-scale Cleaned Chinese Conversation corpus (LCCC) is a large corpus of Chinese conversations.
A rigorous data cleaning pipeline is designed to ensure the quality of the corpus.
This pipeline involves a set of rules and several classifier-based filters.
Noises such as offensive or sensitive words, special symbols, emojis,
grammatically incorrect sentences, and incoherent conversations are filtered.
  • 라이센스 : MIT
  • 버전 : 1.0.0
  • 분할 :
나뉘다
'train' 12007759
  • 특징 :
{
    "dialog": [
        {
            "dtype": "string",
            "id": null,
            "_type": "Value"
        }
    ]
}

베이스

TFDS에 이 데이터세트를 로드하려면 다음 명령어를 사용하세요.

ds = tfds.load('huggingface:lccc/base')
  • 설명 :
LCCC: Large-scale Cleaned Chinese Conversation corpus (LCCC) is a large corpus of Chinese conversations.
A rigorous data cleaning pipeline is designed to ensure the quality of the corpus.
This pipeline involves a set of rules and several classifier-based filters.
Noises such as offensive or sensitive words, special symbols, emojis,
grammatically incorrect sentences, and incoherent conversations are filtered.
  • 라이센스 : MIT
  • 버전 : 1.0.0
  • 분할 :
나뉘다
'test' 10000
'train' 6820506
'validation' 20000
  • 특징 :
{
    "dialog": [
        {
            "dtype": "string",
            "id": null,
            "_type": "Value"
        }
    ]
}