https://github.com/allenai/smashed
SMASHED is a toolkit designed to apply transformations to samples in datasets, such as fields extraction, tokenization, prompting, batching, and more. Supports datasets from Huggingface, torchdata iterables, or simple lists of dictionaries.
dataset
datasets
dict
huggingface
in-context-learning
mappers
natural-language-processing
nlp
pipeline
prefix
prefix-tuning
preprocessing
prompting
pytorch
text
torchdata
transformer
transformers
Added: over 1 year ago - Last Synced: 11 months ago
- Created: July 21, 2022
