Cross-platform IPFS Lite: JS, Android, iOS, and gRPC
tl;dr
- IPFS Lite is a library that provides only the essentials of interacting with the IPFS network: creating new data and getting existing data.
- IPFS Lite is a great choice for applications that want to embed IPFS functionality directly in their code and/or apps.
- Textile is relying on IPFS Lite more and more, so we wanted to contribute some new IPFS Lite libraries to enable the same APIs on all the platforms that Textile runs: JavaScript, Android and iOS.
- We'll continue to develop these libraries as we build around them, but wanted to open them up early so others could try them, give feedback, and help us make them better, faster, and more light-weight.
At Textile, we’ve been working on IPFS in mobile- and browser-based settings for a while now… and we’ve engaged with many developers, trying to optimize how folks can build on top of IPFS and Textile. One thing we’ve noticed again and again is that 99% of the time, developers building on IPFS want to add and get files from the network, with minimal setup or effort. They want IPFS to feel like a simple local datastore. Most of the time, a browser or mobile dApp only needs to be able to add and get small bits of data over the IPFS network. Enter IPFS Lite.
IPFS Lite started as a Go library providing the bare minimum functionality for an IPLD-based application to interact with the IPFS Network. It is a core component of the Merkle-CRDT based distributed datastore as described in the paper by Héctor Sanjuán, Samuli Pöyhtäri and Pedro Teixeira, as well as in the IPFS Cluster project. The API is simple — only a handful of methods by default — and is easy to use/embed in other apps. We found ourselves reaching for IPFS Lite more and more, and decided to make it accessible to more developers.
Today we’re releasing a suite of new IPFS Lite libraries. You can now try IPFS Lite in the browser, on Android, iOS, or desktop (via the original Go library). We’re starting simple, and are hoping to gauge interest and get early feedback with this cross-platform release. Our goal is to provide a highly extensible IPFS implementation that, when 'fully loaded' could support most of the default IPFS core APIs. If you have opinions about what should and should not be included, please let us know or send a PR!
- Browser/Nodejs — https://github.com/textileio/js-ipfs-lite
- Android — https://github.com/textileio/android-ipfs-lite
- iOS — https://github.com/textileio/ios-ipfs-lite
- gRPC — https://github.com/textileio/grpc-ipfs-lite/
What’s next?
You tell us! Seriously, we’d love to hear from you, feel free to reach out over Slack, Twitter, Github, wherever. What kind of use-cases do you have for a light-weight IPFS peer in the browser and on mobile
Libraries in Action
To give you a sense of how you’d use these new libraries in your own applications, here’s a summary of each, with some simple cross-platform examples to give you a taste of the APIs.
Javascript
The js-ipfs-lite implementation started out as a direct port of the original Go implementation, written entirely in Typescript. It has evolved to some degree since then but remains very similar in scope. By default, a lite Peer requires the developer to provide a datastore for storing local IPLD Blocks (this can be an in-memory store for ephemeral peers, a leveldb-based one for persistence, etc.), and a Libp2p host/client. In practice, the library provides a number of helper functions (e.g., setupLibP2PHost) to make initializing a datastore and Libp2p host easier, with defaults appropriate for Nodejs or the browser (based on js-ipfs defaults).
The library is already less than half the size of js-ipfs, and while most core APIs aren’t exposed… a good deal of them are available under the hood. We’ll be working quickly to reduce the size further (we haven’t applied many optimizations at all yet), and expose more of those core APIs as separate, pluggable modules. In the meantime, we’re already finding that reduced load times are leading to a snappier browser experience. If you’re keen to contribute but aren’t versed in Typescript, no biggie, submit a PR anyway, and we’ll help with the types!
Create
let { Peer, BlockStore, setupLibP2PHost } = require('@textile/ipfs-lite')
// Use any interface-datastore compliant store
let { MemoryDatastore } = require('interface-datastore')
let store = new BlockStore(new MemoryDatastore())
let host = await setupLibP2PHost() // Accepts all libp2p config options
let lite = new Peer(store, host)Add a File
// Simple Nodejs example using async/await
let fs = require('fs')
let source = [{
path: 'secret_plans',
content: fs.createReadStream('secret_plans.txt')
}]
let cid = await lite.addFile(source)Get a Node
You can fetch files or explore IPLD Nodes on the network as well. Here, we can explore the Project Apollo Archives:
// Mostly end-to-end browser example using Promises...
let CID = require('cids')
let cid = new CID('QmSnuWmxptJZdLJpKRarxBMS2Ju2oANVrgbr2xWbie9b2D')
setupLibP2PHost().then(host => {
let litePeer = new Peer(store, host)
litePeer.start().then(() => {
litePeer.get(cid).then(block => {
block.Links.forEach(link => {
console.log(link.Name + ': ' + link.Hash.toString())
})
})
})
})Output
README.txt: QmP8jTG1m9GSDJLCbeWhVSVgEzCPPwXRdCRuJtQ5Tz9Kc9
_Metadata.json: QmWXShtJXt6Mw3FH7hVCQvR56xPcaEtSj4YFSGjp2QxA4v
albums: QmUh6QSTxDKX5qoNU1GoogbhTveQQV9JMeQjfFVchAtd5Q
apolloarchivr.py: QmU7gJi6Bz3jrvbuVfB7zzXStLJrTHf6vWh8ZqkCsTGoRC
build_frontend_index.py: QmRSxRRu9AoJ23bxb2pFeoAUFXMAdki7RZu2T7e6zHRdu6
frontend: QmeQtZfwuq6aWRarY9P3L9MWhZ6QTonDe9ahWECGBZjyEJAndroid
For android-ipfs-lite, it's pretty much the same API with simple methods for adding and retrieving files from IPFS. The current implementation doesn't require the developer to provide their own datastore but instead uses a simple embedded datatstore for managing data and nodes added by an application.
Add a file
// Synchronous
File file = openFile("secret_plans");
byte[] bytes = Files.readAllBytes(file.toPath());
String cid = litePeer.addFileSync(bytes);
// Asynchronous
litePeer.addFile(bytes, resultHandler);Get a Node
String cid = "QmSnuWmxptJZdLJpKRarxBMS2Ju2oANVrgbr2xWbie9b2D";
litePeer.getNode(
cid,
new Peer.ResolveNodeHandler() {
public void onNext(Node node) {
List<Link> links = node.getLinksList();
for (int i = 0; i < links.size(); i++) {
String name = links.get(i).getName();
String address = links.get(i).getCid();
logger.log(Level.INFO, name + ": " + address);
}
}
...
}
);Output
README.txt: QmP8jTG1m9GSDJLCbeWhVSVgEzCPPwXRdCRuJtQ5Tz9Kc9
_Metadata.json: QmWXShtJXt6Mw3FH7hVCQvR56xPcaEtSj4YFSGjp2QxA4v
albums: QmUh6QSTxDKX5qoNU1GoogbhTveQQV9JMeQjfFVchAtd5Q
apolloarchivr.py: QmU7gJi6Bz3jrvbuVfB7zzXStLJrTHf6vWh8ZqkCsTGoRC
build_frontend_index.py: QmRSxRRu9AoJ23bxb2pFeoAUFXMAdki7RZu2T7e6zHRdu6
frontend: QmeQtZfwuq6aWRarY9P3L9MWhZ6QTonDe9ahWECGBZjyEJiOS
The iOS implementation is similar to that of Android. You can add and get files to IPFS and query Nodes from the network. There is some remaining work to be done to allow adding of Nodes to the network and to assure that all APIs are using efficient streaming of data and async APIs where applicable.
Add a file
NSString *path = [[NSBundle mainBundle] pathForResource:@"test" ofType:@"jpeg"];
NSInputStream *input = [[NSInputStream alloc] initWithFileAtPath:path];
[IpfsLiteApi.instance addFileWithParams:[[AddParams alloc] init] input:input completion:^(Node *node, NSError *error) {
if (error) {
// handle the error
} else {
// node represents the file now that it's been addded to IPFS
}
}];Get a Node
[IpfsLiteApi.instance getNodeForCid:@"QmSnuWmxptJZdLJpKRarxBMS2Ju2oANVrgbr2xWbie9b2D" completion:^(Node *node, NSError *error) {
for (Link *link in node.linksArray) {
NSLog(@"%@ : %@", link.name, link.cid);
}
}];Output
README.txt : QmP8jTG1m9GSDJLCbeWhVSVgEzCPPwXRdCRuJtQ5Tz9Kc9
_Metadata.json : QmWXShtJXt6Mw3FH7hVCQvR56xPcaEtSj4YFSGjp2QxA4v
albums : QmUh6QSTxDKX5qoNU1GoogbhTveQQV9JMeQjfFVchAtd5Q
apolloarchivr.py : QmU7gJi6Bz3jrvbuVfB7zzXStLJrTHf6vWh8ZqkCsTGoRC
build_frontend_index.py : QmRSxRRu9AoJ23bxb2pFeoAUFXMAdki7RZu2T7e6zHRdu6
frontend : QmeQtZfwuq6aWRarY9P3L9MWhZ6QTonDe9ahWECGBZjyEJ