Summary

In this sprint, we continued to work on core modules for the fil-tools system. Work included the release of miner reputation indices, implementation of end-to-end testing on a local Lotus devnet, CLI for several module APIs, the first implementation of a retrieval API, and more (read more in Sprint Debrief below).

The project is close to providing a drop-in solution for projects that want to build advanced systems and applications on the Filecoin network without requiring them to create custom systems for managing deals, storage, and wallets. The next phase of work will turn our attention to implementing the Fast API, which will allow projects to leverage a complete storage and retrieval pipeline system for their data.

To help onboard new projects, we’ve produced a comprehensive document covering the purpose and design of the fil-tools system.

Background Links 📘

• Overview, architecture, and key concepts.
• Previous progress summary.
• Project blog post.

Next Steps 🍿

• Fast API kick-off. Produce the first MVP of service.
• Provide first use-case in Textile to show using the fil-tools system to store and fetch data over IPFS that is backed by long-term storage on the Filecoin network.

Sprint Debrief

Focus Areas 📐

• Begin work with the go-fil-markets library.
• Use a local devnet for end-to-end testing of the system, in particular, make use of the sectorbuilder mock.
• Develop public APIs/Feeds for Reputation Indices for other projects to consume.
• Finalize analytics/metrics tools (e.g. Grafana) for publishing to the community.
• Add the car file to the Lotus backup archives.

Discussion & Planning Changes 🤔

• Our own understanding of the Fast APIs role and responsibilities has continued to improve. One clear example is that now it is more evident that pieces of the Repair Module are actually core to the Fast API, giving it the ability to repair and renew deals required to keep running.
• We got to know the shift of PieceCID vs. PayloadCID in internals, which will bubble to APIs when feat branch gets into master.

Closed 💥

• Prepare Grafana for community sharing.
• gRPC APIs for slashing, reputation, and ask indices.
• CLI for slashing, reputation, and ask indices (and here).
• Provide setup for running tests on a local devnet.
• Retrieval module.
• Retreival API and CLI.

New Tickets & Questions 👀

• What is delete going to look like in Filecoin?
• If data is stored in more than one miner, Lotus Find API only saves the last miner which we make a deal. It was mentioned that better functionality will be available in late February.
• Currently, knowing which miners store some data is resolved using local Lotus node history (deals Lotus node has made). Will be some global discovery of miners for a particular PayloadCID? It seems that on-chain data won’t be an option since it only stores PieceCID?

Bonus: A fil-tools application

To help us build fil-tools to solve generic developer needs, we are using Textile’s own needs as a one guiding use-case. We are excited to share that progress in Textile is moving very quickly, and this week we will be launching several new Thread based libraries and tools, all of which will soon integrate with Filecoin over fil-tools.

The above is an early preview of our bucket publishing system, which will provide dynamic data buckets available over IPFS and stored on Filecoin through fil-tools.

Contact

• Feel free to join the Textile Slack or join our room on the Filecoin Slack.
• If you would like to discuss our work and how it may fit into your own projects, please reach out through email or setup a quick call here.