مراجع:
برو
برای بارگذاری این مجموعه داده در TFDS از دستور زیر استفاده کنید:
ds = tfds.load('huggingface:code_x_glue_ct_code_to_text/go')
- توضیحات :
CodeXGLUE code-to-text dataset, available at https://github.com/microsoft/CodeXGLUE/tree/main/Code-Text/code-to-text
The dataset we use comes from CodeSearchNet and we filter the dataset as the following:
- Remove examples that codes cannot be parsed into an abstract syntax tree.
- Remove examples that #tokens of documents is < 3 or >256
- Remove examples that documents contain special tokens (e.g. <img ...> or https:...)
- Remove examples that documents are not English.
- مجوز : مجوز شناخته شده ای وجود ندارد
- نسخه : 0.0.0
- تقسیم ها :
تقسیم کنید | نمونه ها |
---|---|
'test' | 8122 |
'train' | 167288 |
'validation' | 7325 |
- ویژگی ها :
{
"id": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"repo": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"path": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"func_name": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"original_string": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"language": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"code": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"code_tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"docstring": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"docstring_tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"sha": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"url": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
جاوا
برای بارگذاری این مجموعه داده در TFDS از دستور زیر استفاده کنید:
ds = tfds.load('huggingface:code_x_glue_ct_code_to_text/java')
- توضیحات :
CodeXGLUE code-to-text dataset, available at https://github.com/microsoft/CodeXGLUE/tree/main/Code-Text/code-to-text
The dataset we use comes from CodeSearchNet and we filter the dataset as the following:
- Remove examples that codes cannot be parsed into an abstract syntax tree.
- Remove examples that #tokens of documents is < 3 or >256
- Remove examples that documents contain special tokens (e.g. <img ...> or https:...)
- Remove examples that documents are not English.
- مجوز : مجوز شناخته شده ای وجود ندارد
- نسخه : 0.0.0
- تقسیم ها :
تقسیم کنید | نمونه ها |
---|---|
'test' | 10955 |
'train' | 164923 |
'validation' | 5183 |
- ویژگی ها :
{
"id": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"repo": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"path": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"func_name": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"original_string": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"language": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"code": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"code_tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"docstring": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"docstring_tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"sha": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"url": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
جاوا اسکریپت
برای بارگذاری این مجموعه داده در TFDS از دستور زیر استفاده کنید:
ds = tfds.load('huggingface:code_x_glue_ct_code_to_text/javascript')
- توضیحات :
CodeXGLUE code-to-text dataset, available at https://github.com/microsoft/CodeXGLUE/tree/main/Code-Text/code-to-text
The dataset we use comes from CodeSearchNet and we filter the dataset as the following:
- Remove examples that codes cannot be parsed into an abstract syntax tree.
- Remove examples that #tokens of documents is < 3 or >256
- Remove examples that documents contain special tokens (e.g. <img ...> or https:...)
- Remove examples that documents are not English.
- مجوز : مجوز شناخته شده ای وجود ندارد
- نسخه : 0.0.0
- تقسیم ها :
تقسیم کنید | نمونه ها |
---|---|
'test' | 3291 |
'train' | 58025 |
'validation' | 3885 |
- ویژگی ها :
{
"id": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"repo": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"path": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"func_name": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"original_string": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"language": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"code": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"code_tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"docstring": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"docstring_tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"sha": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"url": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
php
برای بارگذاری این مجموعه داده در TFDS از دستور زیر استفاده کنید:
ds = tfds.load('huggingface:code_x_glue_ct_code_to_text/php')
- توضیحات :
CodeXGLUE code-to-text dataset, available at https://github.com/microsoft/CodeXGLUE/tree/main/Code-Text/code-to-text
The dataset we use comes from CodeSearchNet and we filter the dataset as the following:
- Remove examples that codes cannot be parsed into an abstract syntax tree.
- Remove examples that #tokens of documents is < 3 or >256
- Remove examples that documents contain special tokens (e.g. <img ...> or https:...)
- Remove examples that documents are not English.
- مجوز : مجوز شناخته شده ای وجود ندارد
- نسخه : 0.0.0
- تقسیم ها :
تقسیم کنید | نمونه ها |
---|---|
'test' | 14014 |
'train' | 241241 |
'validation' | 12982 |
- ویژگی ها :
{
"id": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"repo": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"path": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"func_name": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"original_string": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"language": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"code": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"code_tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"docstring": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"docstring_tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"sha": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"url": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
پایتون
برای بارگذاری این مجموعه داده در TFDS از دستور زیر استفاده کنید:
ds = tfds.load('huggingface:code_x_glue_ct_code_to_text/python')
- توضیحات :
CodeXGLUE code-to-text dataset, available at https://github.com/microsoft/CodeXGLUE/tree/main/Code-Text/code-to-text
The dataset we use comes from CodeSearchNet and we filter the dataset as the following:
- Remove examples that codes cannot be parsed into an abstract syntax tree.
- Remove examples that #tokens of documents is < 3 or >256
- Remove examples that documents contain special tokens (e.g. <img ...> or https:...)
- Remove examples that documents are not English.
- مجوز : مجوز شناخته شده ای وجود ندارد
- نسخه : 0.0.0
- تقسیم ها :
تقسیم کنید | نمونه ها |
---|---|
'test' | 14918 |
'train' | 251820 |
'validation' | 13914 |
- ویژگی ها :
{
"id": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"repo": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"path": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"func_name": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"original_string": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"language": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"code": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"code_tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"docstring": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"docstring_tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"sha": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"url": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
یاقوت سرخ
برای بارگذاری این مجموعه داده در TFDS از دستور زیر استفاده کنید:
ds = tfds.load('huggingface:code_x_glue_ct_code_to_text/ruby')
- توضیحات :
CodeXGLUE code-to-text dataset, available at https://github.com/microsoft/CodeXGLUE/tree/main/Code-Text/code-to-text
The dataset we use comes from CodeSearchNet and we filter the dataset as the following:
- Remove examples that codes cannot be parsed into an abstract syntax tree.
- Remove examples that #tokens of documents is < 3 or >256
- Remove examples that documents contain special tokens (e.g. <img ...> or https:...)
- Remove examples that documents are not English.
- مجوز : مجوز شناخته شده ای وجود ندارد
- نسخه : 0.0.0
- تقسیم ها :
تقسیم کنید | نمونه ها |
---|---|
'test' | 1261 |
'train' | 24927 |
'validation' | 1400 |
- ویژگی ها :
{
"id": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"repo": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"path": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"func_name": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"original_string": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"language": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"code": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"code_tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"docstring": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"docstring_tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"sha": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"url": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}