pybikes
Provides a set of tools to scrape bike sharing data from different websites and APIs, thus providing a coherent and generalized set of classes and methods to access this sort of information.
https://github.com/eskerda/pybikes
Category: Consumption
Sub Category: Mobility and Transportation
Keywords from Contributors
bike-sharing cycling bycycle cities shared-mobility bikesharing carshare mobility gbfs scooter-sharing
Last synced: about 10 hours ago
JSON representation
Repository metadata
bike sharing + python = pybikes
- Host: GitHub
- URL: https://github.com/eskerda/pybikes
- Owner: eskerda
- License: lgpl-3.0
- Created: 2010-07-16T10:07:59.000Z (almost 15 years ago)
- Default Branch: master
- Last Pushed: 2025-04-20T18:37:09.000Z (7 days ago)
- Last Synced: 2025-04-21T12:07:12.748Z (7 days ago)
- Language: Python
- Homepage: https://citybik.es
- Size: 2.42 MB
- Stars: 578
- Watchers: 32
- Forks: 169
- Open Issues: 24
- Releases: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.txt
README.md
pybikes provides a set of tools to scrape bike sharing data from different
websites and APIs, thus providing a coherent and generalized set of classes
and methods to access this sort of information.
The library is distributed and intended mainly for statistics and data
sharing projects. More importantly, it powers the CityBikes project, and
is composed of a set of classes and a pack of data files that provide instances
for all different systems.
Installation
Install directly from GitHub:
pip install git+https://github.com/eskerda/pybikes.git
Or after downloading/cloning the source:
python setup.py install
The following dependencies are required (example using Ubuntu package manager):
sudo apt-get install python
sudo apt-get install python-setuptools
sudo apt-get install libxml2 libxml2-dev libxslt1-dev libgeos-dev
Usage
>>> import pybikes
# Capital BikeShare instantiation data is in bixi.json file
>>> capital_bikeshare = pybikes.get('capital-bikeshare')
# The instance contains all possible metadata regarding this system
>>> print(capital_bikeshare.meta)
{
'name': 'Capital BikeShare',
'city': 'Washington, DC - Arlington, VA',
'longitude': -77.0363658,
'system': 'Bixi',
'company': ['PBSC'],
'country': 'USA',
'latitude': 38.8951118
}
# The update method retrieves the list of stations
>>> print(len(capital_bikeshare.stations))
0
>>> capital_bikeshare.update()
>>> print(len(capital_bikeshare.stations))
191
>>> print(capital_bikeshare.stations[0])
--- 31000 - 20th & Bell St ---
bikes: 7
free: 4
latlng: 38.8561,-77.0512
extra: {
'installed': True,
'uid': 1,
'locked': False,
'removalDate': '',
'installDate': '1316059200000',
'terminalName': '31000',
'temporary': False,
'name': '20th & Bell St',
'latestUpdateTime': '1353454305589'
}
Some systems might require an API key to work (for instance, Cyclocity). In
these cases, the instance factory can take an extra API key parameter.
>>> key = "This is not an API key"
>>> dublinbikes = pybikes.get('dublinbikes', key)
Note that pybikes works as an instance factory and, choicely, instances can be
generated by passing the right arguments to the desired class
>>> from pybikes.cyclocity import BixiSystem
>>> capital_bikeshare = BixiSystem(
tag = 'foo_tag',
root_url = 'http://capitalbikeshare.com/data/stations/',
meta = {'foo':'bar'}
)
The way information is retrieved can be tweaked using the PyBikesScraper class
included on the utils module thus allowing session reusing and niceties such as
using a proxy. This class uses Requests module internally.
>>> scraper = pybikes.utils.PyBikesScraper()
>>> scraper.enableProxy()
>>> scraper.setProxies({
"http" : "127.0.0.1:8118",
"https": "127.0.0.1:8118"
})
>>> scraper.setUserAgent("Walrus™ v3.0")
>>> scraper.headers['Foo'] = 'bar'
>>> capital_bikeshare.update(scraper)
Tests
Tests are separated between unit tests and integration tests with the different
sources supported.
To run unit tests simply
make test
To run integration tests
make test-update
Note that some systems require authorization keys, tests expect these to be
set as environment variables like:
PYBIKES_CYCLOCITY='some-api-key'
PYBIKES_DEUTSCHEBAHN_CLIENT_ID='some-client-id'
PYBIKES_DEUTSCHEBAHN_CLIENT_SECRET='some-client-secret'
# or if using an .env file
# source .env
make test-update
This project uses pytest for tests. Test a particular network by passing a
filter expresson
pytest -k bicing
pytest -k gbfs
To speed up tests execution, install pytest-xdist to specify the number of
CPUs to use
pytest -k gbfs -n auto
To use Makefile steps and pass along pytest arguments, append to the T_FLAGS
variable
make test-update T_FLAGS+='-n 10 -k gbfs'
Integration tests can generate a json report file with all extracted data stored
as geojson. Using this json report file, further useful reports can be generated
like a summary of the overall health of the library or a map visualization of
all the information.
For more information on reports see utils/README.md
Development
We welcome contributions from the community! The best place to get started is
by diving into the codebase or checking the issues list.
Join our developer community on Matrix: #citybikes:matrix.org
Owner metadata
- Name: eskerda
- Login: eskerda
- Email:
- Kind: user
- Description:
- Website: https://eskerda.com
- Location: Barcelona
- Twitter:
- Company:
- Icon url: https://avatars.githubusercontent.com/u/208952?u=9331dd647d98b898ca4dd4fe20c2c9923e3a6df6&v=4
- Repositories: 45
- Last ynced at: 2024-06-11T15:58:36.815Z
- Profile URL: https://github.com/eskerda
GitHub Events
Total
- Issues event: 18
- Watch event: 26
- Delete event: 14
- Issue comment event: 42
- Push event: 54
- Pull request event: 69
- Pull request review event: 7
- Pull request review comment event: 9
- Fork event: 6
- Create event: 13
Last Year
- Issues event: 18
- Watch event: 26
- Delete event: 14
- Issue comment event: 42
- Push event: 54
- Pull request event: 69
- Pull request review event: 7
- Pull request review comment event: 9
- Fork event: 6
- Create event: 13
Committers metadata
Last synced: 8 days ago
Total Commits: 1,283
Total Committers: 104
Avg Commits per committer: 12.337
Development Distribution Score (DDS): 0.436
Commits in past year: 116
Committers in past year: 8
Avg Commits per committer in past year: 14.5
Development Distribution Score (DDS) in past year: 0.388
Name | Commits | |
---|---|---|
eskerda | e****a@g****m | 723 |
Martín González Gómez | m****z@n****m | 126 |
eduardom | e****i@g****m | 64 |
Altonss | 6****s | 53 |
Jean-Luc Tibaux | e****c@o****e | 20 |
Adam Danielczyk | 1****k | 15 |
Benjamin Stolte | b****e@i****e | 14 |
felix.gravet | f****t@g****r | 14 |
bparmentier | d****v@b****e | 13 |
Ortwin Gentz | g****z@f****m | 13 |
Emmanuel Haguet | M****0 | 10 |
Lev Aronsky | a****y@g****m | 9 |
Hugo | h****k | 9 |
Martin Ichilevici de Oliveira | m****a@g****m | 7 |
Mauricio Maia | m****a@g****m | 7 |
Olav Alexander Mjelde | o****e@g****m | 7 |
R0nd | r****1@g****m | 7 |
bcaller | b****r | 7 |
Maximilian Richt | m****i@r****e | 6 |
Anastasios Mag | a****s@m****m | 6 |
Ulrich Stroetz | u****z@g****m | 5 |
João Pereira | 6****p | 5 |
Gabor Szathmari | g****i@g****m | 5 |
François FERREIRA DE SOUSA | f****s@f****r | 4 |
thesebas | t****s@t****t | 4 |
Igor Matos | i****s@l****m | 4 |
Daniel Korp | d****p@b****t | 4 |
thomas | t****r@o****r | 4 |
Kostiantyn Plakhotia | k****l | 4 |
Snarkorel | s****7@m****u | 4 |
and 74 more... |
Committer domains:
- iki.fi: 2
- bikecitizens.net: 2
- kai.gs: 1
- ludoscity.com: 1
- kreitschmann.de: 1
- lepagnot.fr: 1
- berriart.com: 1
- schenkel.net: 1
- rome2rio.com: 1
- me.com: 1
- freemap.sk: 1
- mail.ru: 1
- ovh.fr: 1
- liferay.com: 1
- thesebas.net: 1
- fdesousa.fr: 1
- mageirias.com: 1
- richt.name: 1
- futuretap.com: 1
- brunoparmentier.be: 1
- geovelo.fr: 1
- innoz.de: 1
- outlook.de: 1
- nytimes.com: 1
- reallymy.email: 1
- gogn.in: 1
- violoncello.ch: 1
- nextbike.net: 1
- idupree.com: 1
- users.noreply.gitlab.com: 1
- uni-wuerzburg.de: 1
- njoylab.com: 1
- zickler.us: 1
- mac.com: 1
- lucas-smith.co.uk: 1
- iomartin.net: 1
- matiascontilde.com: 1
- rasumi.net: 1
- xuaps.com: 1
- robbiet.us: 1
- phyks.me: 1
- ugedal.com: 1
- outlook.it: 1
- mit.edu: 1
- geoinformationsdienst.de: 1
- altlinux.ru: 1
- gmx.de: 1
Issue and Pull Request metadata
Last synced: 1 day ago
Total issues: 212
Total pull requests: 618
Average time to close issues: over 1 year
Average time to close pull requests: about 2 months
Total issue authors: 92
Total pull request authors: 111
Average comments per issue: 2.56
Average comments per pull request: 1.37
Merged pull request: 543
Bot issues: 0
Bot pull requests: 1
Past year issues: 24
Past year pull requests: 80
Past year average time to close issues: 16 days
Past year average time to close pull requests: 6 days
Past year issue authors: 9
Past year pull request authors: 9
Past year average comments per issue: 1.54
Past year average comments per pull request: 0.8
Past year merged pull request: 73
Past year bot issues: 0
Past year bot pull requests: 0
Top Issue Authors
- eskerda (63)
- Altonss (20)
- futuretap (6)
- Manu1400 (5)
- nixta (4)
- tuxayo (4)
- f8full (3)
- MichalPP (3)
- Gabriel44200 (3)
- homtec (3)
- arrobaraujo (3)
- bcaller (3)
- habi (3)
- rolinger (3)
- andrewshadura (2)
Top Pull Request Authors
- eskerda (179)
- martgnz (112)
- edumucelli (47)
- Altonss (44)
- Ephtolens (16)
- BenSto (14)
- Manu1400 (13)
- adamdanielczyk (10)
- futuretap (9)
- eUgEntOptIc44 (8)
- bparmentier (7)
- anastmag (7)
- bcaller (5)
- kospl (4)
- robbi5 (4)
Top Issue Labels
- accepted (2)
- experimental (2)
- feature (1)
- bug (1)
Top Pull Request Labels
- dependencies (1)
Package metadata
- Total packages: 1
- Total downloads: unknown
- Total dependent packages: 0
- Total dependent repositories: 0
- Total versions: 5
- Total maintainers: 1
pypi.org: pybikes
A python library for scraping bike sharing data
- Homepage: https://www.citybik.es
- Documentation: https://docs.citybik.es
- Licenses: lgpl-3.0
- Latest release: 1.0.3 (published 2 days ago)
- Last Synced: 2025-04-26T14:39:54.550Z (1 day ago)
- Versions: 5
- Dependent Packages: 0
- Dependent Repositories: 0
-
Rankings:
- Dependent packages count: 9.251%
- Average: 30.681%
- Dependent repos count: 52.11%
- Maintainers (1)
Dependencies
- flake8 *
- jinja2 *
- mock *
- pytest *
- pytest-json-report *
- pytest-xdist *
- cssselect >=0.9
- lxml *
- requests >=2.20.0
- shapely >=1.5.13
- actions/checkout v3 composite
- actions/setup-python v3 composite
- MatteoH2O1999/setup-python v1 composite
- actions/checkout v3 composite
Score: 11.044648344450193