Referências:
sou
Use o seguinte comando para carregar este conjunto de dados no TFDS:
ds = tfds.load('huggingface:cc100/am')
- Descrição :
This corpus is an attempt to recreate the dataset used for training XLM-R. This corpus comprises of monolingual data for 100+ languages and also includes data for romanized languages (indicated by *_rom). This was constructed using the urls and paragraph indices provided by the CC-Net repository by processing January-December 2018 Commoncrawl snapshots. Each file comprises of documents separated by double-newlines and paragraphs within the same document separated by a newline. The data is generated using the open source CC-Net repository. No claims of intellectual property are made on the work of preparation of the corpus.
- Licença : Nenhuma licença conhecida
- Versão : 1.0.0
- Divisões :
Dividir | Exemplos |
---|---|
'train' | 3124561 |
- Características :
{
"id": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"text": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
senhor
Use o seguinte comando para carregar este conjunto de dados no TFDS:
ds = tfds.load('huggingface:cc100/sr')
- Descrição :
This corpus is an attempt to recreate the dataset used for training XLM-R. This corpus comprises of monolingual data for 100+ languages and also includes data for romanized languages (indicated by *_rom). This was constructed using the urls and paragraph indices provided by the CC-Net repository by processing January-December 2018 Commoncrawl snapshots. Each file comprises of documents separated by double-newlines and paragraphs within the same document separated by a newline. The data is generated using the open source CC-Net repository. No claims of intellectual property are made on the work of preparation of the corpus.
- Licença : Nenhuma licença conhecida
- Versão : 1.0.0
- Divisões :
Dividir | Exemplos |
---|---|
'train' | 35747957 |
- Características :
{
"id": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"text": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
ka
Use o seguinte comando para carregar este conjunto de dados no TFDS:
ds = tfds.load('huggingface:cc100/ka')
- Descrição :
This corpus is an attempt to recreate the dataset used for training XLM-R. This corpus comprises of monolingual data for 100+ languages and also includes data for romanized languages (indicated by *_rom). This was constructed using the urls and paragraph indices provided by the CC-Net repository by processing January-December 2018 Commoncrawl snapshots. Each file comprises of documents separated by double-newlines and paragraphs within the same document separated by a newline. The data is generated using the open source CC-Net repository. No claims of intellectual property are made on the work of preparation of the corpus.
- Licença : Nenhuma licença conhecida
- Versão : 1.0.0
- Divisões :
Dividir | Exemplos |
---|---|
'train' | 31708119 |
- Características :
{
"id": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"text": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}