ccaligned_multilingual

Tài liệu tham khảo:

tài liệu-zz_TR

Sử dụng lệnh sau để tải tập dữ liệu này trong TFDS:

ds = tfds.load('huggingface:ccaligned_multilingual/documents-zz_TR')
  • Sự miêu tả :
CCAligned consists of parallel or comparable web-document pairs in 137 languages aligned with English. These web-document pairs were constructed by performing language identification on raw web-documents, and ensuring corresponding language codes were corresponding in the URLs of web documents. This pattern matching approach yielded more than 100 million aligned documents paired with English. Recognizing that each English document was often aligned to mulitple documents in different target language, we can join on English documents to obtain aligned documents that directly pair two non-English documents (e.g., Arabic-French).
  • Giấy phép : Không có giấy phép được biết đến
  • Phiên bản : 1.0.0
  • Chia tách :
Tách ra Ví dụ
'train' 41
  • Đặc trưng : 
{
    "Domain": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    },
    "Source_URL": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    },
    "Target_URL": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    },
    "translation": {
        "languages": [
            "en_XX",
            "zz_TR"
        ],
        "id": null,
        "_type": "Translation"
    }
}

câu-zz_TR

Sử dụng lệnh sau để tải tập dữ liệu này trong TFDS:

ds = tfds.load('huggingface:ccaligned_multilingual/sentences-zz_TR')
  • Sự miêu tả :
CCAligned consists of parallel or comparable web-document pairs in 137 languages aligned with English. These web-document pairs were constructed by performing language identification on raw web-documents, and ensuring corresponding language codes were corresponding in the URLs of web documents. This pattern matching approach yielded more than 100 million aligned documents paired with English. Recognizing that each English document was often aligned to mulitple documents in different target language, we can join on English documents to obtain aligned documents that directly pair two non-English documents (e.g., Arabic-French).
  • Giấy phép : Không có giấy phép được biết đến
  • Phiên bản : 1.0.0
  • Chia tách :
Tách ra Ví dụ
'train' 34
  • Đặc trưng : 
{
    "translation": {
        "languages": [
            "en_XX",
            "zz_TR"
        ],
        "id": null,
        "_type": "Translation"
    },
    "LASER_similarity": {
        "dtype": "float32",
        "id": null,
        "_type": "Value"
    }
}

tài liệu-tz_MA

Sử dụng lệnh sau để tải tập dữ liệu này trong TFDS:

ds = tfds.load('huggingface:ccaligned_multilingual/documents-tz_MA')
  • Sự miêu tả :
CCAligned consists of parallel or comparable web-document pairs in 137 languages aligned with English. These web-document pairs were constructed by performing language identification on raw web-documents, and ensuring corresponding language codes were corresponding in the URLs of web documents. This pattern matching approach yielded more than 100 million aligned documents paired with English. Recognizing that each English document was often aligned to mulitple documents in different target language, we can join on English documents to obtain aligned documents that directly pair two non-English documents (e.g., Arabic-French).
  • Giấy phép : Không có giấy phép được biết đến
  • Phiên bản : 1.0.0
  • Chia tách :
Tách ra Ví dụ
'train' 4
  • Đặc trưng : 
{
    "Domain": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    },
    "Source_URL": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    },
    "Target_URL": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    },
    "translation": {
        "languages": [
            "en_XX",
            "tz_MA"
        ],
        "id": null,
        "_type": "Translation"
    }
}

câu-tz_MA

Sử dụng lệnh sau để tải tập dữ liệu này trong TFDS:

ds = tfds.load('huggingface:ccaligned_multilingual/sentences-tz_MA')
  • Sự miêu tả :
CCAligned consists of parallel or comparable web-document pairs in 137 languages aligned with English. These web-document pairs were constructed by performing language identification on raw web-documents, and ensuring corresponding language codes were corresponding in the URLs of web documents. This pattern matching approach yielded more than 100 million aligned documents paired with English. Recognizing that each English document was often aligned to mulitple documents in different target language, we can join on English documents to obtain aligned documents that directly pair two non-English documents (e.g., Arabic-French).
  • Giấy phép : Không có giấy phép được biết đến
  • Phiên bản : 1.0.0
  • Chia tách :
Tách ra Ví dụ
'train' 33
  • Đặc trưng : 
{
    "translation": {
        "languages": [
            "en_XX",
            "tz_MA"
        ],
        "id": null,
        "_type": "Translation"
    },
    "LASER_similarity": {
        "dtype": "float32",
        "id": null,
        "_type": "Value"
    }
}

tài liệu-ak_GH

Sử dụng lệnh sau để tải tập dữ liệu này trong TFDS:

ds = tfds.load('huggingface:ccaligned_multilingual/documents-ak_GH')
  • Sự miêu tả :
CCAligned consists of parallel or comparable web-document pairs in 137 languages aligned with English. These web-document pairs were constructed by performing language identification on raw web-documents, and ensuring corresponding language codes were corresponding in the URLs of web documents. This pattern matching approach yielded more than 100 million aligned documents paired with English. Recognizing that each English document was often aligned to mulitple documents in different target language, we can join on English documents to obtain aligned documents that directly pair two non-English documents (e.g., Arabic-French).
  • Giấy phép : Không có giấy phép được biết đến
  • Phiên bản : 1.0.0
  • Chia tách :
Tách ra Ví dụ
'train' 249
  • Đặc trưng : 
{
    "Domain": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    },
    "Source_URL": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    },
    "Target_URL": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    },
    "translation": {
        "languages": [
            "en_XX",
            "ak_GH"
        ],
        "id": null,
        "_type": "Translation"
    }
}

câu-ak_GH

Sử dụng lệnh sau để tải tập dữ liệu này trong TFDS:

ds = tfds.load('huggingface:ccaligned_multilingual/sentences-ak_GH')
  • Sự miêu tả :
CCAligned consists of parallel or comparable web-document pairs in 137 languages aligned with English. These web-document pairs were constructed by performing language identification on raw web-documents, and ensuring corresponding language codes were corresponding in the URLs of web documents. This pattern matching approach yielded more than 100 million aligned documents paired with English. Recognizing that each English document was often aligned to mulitple documents in different target language, we can join on English documents to obtain aligned documents that directly pair two non-English documents (e.g., Arabic-French).
  • Giấy phép : Không có giấy phép được biết đến
  • Phiên bản : 1.0.0
  • Chia tách :
Tách ra Ví dụ
'train' 478
  • Đặc trưng :
{
    "translation": {
        "languages": [
            "en_XX",
            "ak_GH"
        ],
        "id": null,
        "_type": "Translation"
    },
    "LASER_similarity": {
        "dtype": "float32",
        "id": null,
        "_type": "Value"
    }
}