https://github.com/allenai/smashed

SMASHED is a toolkit designed to apply transformations to samples in datasets, such as fields extraction, tokenization, prompting, batching, and more. Supports datasets from Huggingface, torchdata iterables, or simple lists of dictionaries.
https://github.com/allenai/smashed

Keywords

dataset datasets dict huggingface in-context-learning mappers natural-language-processing nlp pipeline prefix prefix-tuning preprocessing prompting pytorch text torchdata transformer transformers

Keywords from Contributors

archiving measur observation generic compose large-language-models animals conversion language-model optimize

Last synced: 11 months ago
JSON representation

Acceptance Criteria

• Revelant topics? true
• External users? true
• Open source license? true
• Active? true
• Fork? false

👍 0

👎 0

Repository metadata

SMASHED is a toolkit designed to apply transformations to samples in datasets, such as fields extraction, tokenization, prompting, batching, and more. Supports datasets from Huggingface, torchdata iterables, or simple lists of dictionaries.

• Host: GitHub
• URL: https://github.com/allenai/smashed
• Owner: allenai
• License: apache-2.0
• Created: 2022-07-21T17:11:17.000Z (almost 3 years ago)
• Default Branch: main
• Last Pushed: 2023-10-02T12:23:22.000Z (over 1 year ago)
• Last Synced: 2024-05-03T07:15:29.777Z (about 1 year ago)
• Topics: dataset, datasets, dict, huggingface, in-context-learning, mappers, natural-language-processing, nlp, pipeline, prefix, prefix-tuning, preprocessing, prompting, pytorch, text, torchdata, transformer, transformers
• Language: Python
• Homepage: https://pypi.org/project/smashed
• Size: 4.55 MB
• Stars: 30
• Watchers: 6
• Forks: 3
• Open Issues: 6
• Releases: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

---

Owner metadata

• Name: AI2
• Login: allenai
• Email: [email protected]
• Kind: organization
• Description:
• Website: http://www.allenai.org
• Location: Seattle, WA
• Twitter:
• Company:
• Icon url: https://avatars.githubusercontent.com/u/5667695?v=4
• Repositories: 454
• Last ynced at: 2024-04-14T22:06:46.803Z
• Profile URL: https://github.com/allenai

---

GitHub Events

Total

• Fork event: 3
• Create event: 101
• Release event: 39
• Issues event: 1
• Watch event: 32
• Delete event: 44
• Member event: 1
• Issue comment event: 11
• Public event: 1
• Push event: 264
• Pull request review comment event: 7
• Pull request review event: 20
• Pull request event: 114

Last Year

• Create event: 8
• Delete event: 2
• Issue comment event: 1
• Pull request event: 9
• Pull request review event: 1
• Push event: 7
• Release event: 1
• Watch event: 7

---

Committers metadata

Last synced: about 1 year ago

Name	Email	Commits
Luca Soldaini	l****s@a****g	100
Luca Soldaini	l****a@s****t	21
kyleclo	k****o@u****u	14
dependabot[bot]	4****]	7
Maksym Del	m****u@g****m	2
Ben Newman	b****9	1

Committer domains:

• uw.edu: 1
• soldaini.net: 1
• allenai.org: 1

---

Issue and Pull Request metadata

Last synced: 12 months ago

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/allenai/smashed

---

---

---

Package metadata

• Total packages: 1
• Total downloads:
  • pypi: 11,380 last-month
• Total dependent packages: 2
• Total dependent repositories: 1
• Total versions: 64
• Total maintainers: 2

---

Dependencies

Score: 14.715330386182005