Skip to content
This repository has been archived by the owner on Dec 10, 2017. It is now read-only.
/ Herd Public archive

A single-command bittorrent distribution system, based on Twitter's Murder

License

Notifications You must be signed in to change notification settings

russss/Herd

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

82 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

DEPRECATED PROJECT: I (@russss) am no longer maintaining this code - in fact I've never actually deployed it in production myself, although others have. I won't be accepting any more pull requests for this project. If you're maintaining a fork, let me know and I'll link to it.

About

Herd is a torrent-based file distribution system based on Murder. It allows for quick and easy transfer of small and large files. You can probably use it for other things too.

Herd requires no extra Python modules and includes everything needed for destinations including its own (lightly modified) copy of BitTornado.

Herd has been updated from its fork Horde which removed limitations around large file transfers, peer seeding, and python integration. Herd now has the same capabilities as Horde in that regard.

Differences from Murder

Murder was Twitter's original BitTorrent-based file distribution system. It's pretty dependent on Capistrano and requires that a separate tracker process is started before you run the deploy task.

Herd spawns its own tracker in the background which makes it really trivial to integrate into whatever deployment system you like.

Requirements

Herd needs Python > 2.5 and argparse. If you're using python 2.7+ nothing else is needed as argparse was added to the standard library.

Argparse, if needed, can be installed with easy_install or pip:

pip install argparse
easy_install argparse

Herd also requires that key based passwordless authentication is enabled on all of your target destinations.

Install

git clone https://github.com/russss/Herd
cd Herd && sudo python setup.py install

Herd also supports pypi_server hosting.

Usage

With a hosts file that includes a list of the hosts you want to copy to:

herd myfile.tar.gz /path/to/destination.tar.gz hosts_file

Using a hosts list that is a single string comma separated:

herd myfile.tar.gz /path/to/destination.tar.gz --hostlist "host1,host2,host3"

Reading the host list from stdin.

cat /tmp/hosts_file | herd myfile.tar.gz /path/to/destination.tar.gz hosts_file

More options:

herd --help

Python Integration

Herd can also be imported as a python module. This makes integration into existing projects easy. One would simply need to:

import herd.herd as herd
herd.run_with_opts('localfile', 'remotefile', hostlist='server1,server2')

Credits

About

A single-command bittorrent distribution system, based on Twitter's Murder

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages