Data

Tools

Indexes

Applications

Experiments

Open Sustainable Technology

https://github.com/hmunachi/nanodl

A Jax-based library for designing and training transformer models from scratch.
attention attention-mechanism deep-learning distributed-training flax gpt jax llama machine-learning mistral nlp transformer
Added: about 1 year ago - Last Synced: 11 months ago - Created: August 22, 2023

  • Relevant topics? true
  • External users? true
  • Open source license? true
  • Active? true
  • Fork? false
  • Main Language: Python
  • Commits:
  • Committers:
  • Issues: 9
  • Pull Requests: 15
  • Owner: HMUNACHI
  • Stars: 256
  • Forks: 10
  • Packages: 1
  • Downloads: 103
https://github.com/kyegomez/gats

Implementation of GATS from the paper: "GATS: Gather-Attend-Scatter" in pytorch and zeta
ai attention attention-is-all-you-need attention-mechanism gpt4 llama ml multi-modal multi-modality multimodal open-source
Added: over 1 year ago - Last Synced: 11 months ago - Created: January 18, 2024

  • Relevant topics? true
  • External users? true
  • Open source license? true
  • Active? true
  • Fork? false
  • Main Language: Python
  • Commits: 12
  • Committers: 2
  • Issues: 0
  • Pull Requests: 3
  • Owner: kyegomez
  • Stars: 8
  • Forks: 0
  • Packages: 1
  • Downloads: 24
https://github.com/kyegomez/mmca

The open source community's implementation of the all-new Multi-Modal Causal Attention from "DeepSpeed-VisualChat: Multi-Round Multi-Image Interleave Chat via Multi-Modal Causal Attention"
artificial-intelligence attention attention-is-all-you-need attention-mechanism gpt4 multimodal multimodality neural-network neuralnetwork opensource-library opensourceforgood
Added: over 1 year ago - Last Synced: 11 months ago - Created: September 27, 2023

  • Relevant topics? true
  • External users? true
  • Open source license? true
  • Active? true
  • Fork? false
  • Main Language: Python
  • Commits: 15
  • Committers: 3
  • Issues: 1
  • Pull Requests: 12
  • Owner: kyegomez
  • Stars: 9
  • Forks: 0
  • Packages: 1
  • Downloads: 20
https://github.com/kyegomez/attnwithconvolutions

Interleaved Attention's with convolutions for text modeling
artificial-intelligence attention attention-mechanism convolution convolutional-neural-networks gpt4 llama machine-learning machine-learning-algorithms
Added: over 1 year ago - Last Synced: 11 months ago - Created: November 28, 2023

  • Relevant topics? true
  • External users? true
  • Open source license? true
  • Active? true
  • Fork? false
  • Main Language: Python
  • Commits: 10
  • Committers: 2
  • Issues: 1
  • Pull Requests: 10
  • Owner: kyegomez
  • Stars: 5
  • Forks: 0
  • Packages: 1
  • Downloads: 11