Weeknotes: January 8, 2024

A weekly digest of progress, updates, and insights from Tableland. This week, we explore web3.storage & storing IPFS references in tables, the Studio CLI's cost estimation feature, and more.

Begin transmission…

Persisting data with web3.storage & Tableland tables

by Dan Buchholz

Tableland naturally imposes limits (1 kb cell maximum) on the size of data you can store in a table. If you need to store larger pieces media, you can use IPFS to store the data and then reference it in your table—and web3.storage makes it easy to enable persistence guarantees by creating Filecoin deals for your content.

You can start by installing the Tableland SDK and web.storage client:

npm install @tableland/sdk @web3-storage/w3up-client

Then, set up your Tableland database connection. You can choose whatever chain you'd like, but Filecoin's FEVM is a perfect fit. The example below shows how you'd connection to the Filecoin Calibration testnet (with Hardhat private key):

import { Database } from "@tableland/sdk"; import { Wallet, getDefaultProvider } from "ethers"; import { create } from "@web3-storage/w3up-client"; const privateKey = "59c6995e998f97a5a0044966f0945389dc9e86dae88c7a8412f4603b6b78690d"; // Replace with your private key const wallet = new Wallet(privateKey); const provider = getDefaultProvider( "https://api.calibration.node.glif.io/rpc/v1" // Connect to Filecoin Calibration ); const signer = wallet.connect(provider);

Now, let's set up web3.storage ("w3s"). You'll first need to create an account on the platform, and then you must go through the process of logging in and setting up a "space" to store the data.

const space = await client.createSpace("tableland"); const w3sAccount = await client.login("[email protected]"); // Replace with your w3s account's email address await w3sAccount.provision(space.did()); await space.createRecovery(w3sAccount.did()); await space.save(); await client.setCurrentSpace(space.did());

When you call client.login() for the first time, you'll be prompted to authenticate with your email address—you'll have to go to your email inbox and click the verification. Then, you can upload your data to the space. Once this is set up, you can create a table and then use w3s to store a "large" piece of data (e.g., files).

// Create a database connection and create a table const db = new Database({ signer }); const { meta: create } = await db .prepare(`CREATE TABLE my_table (id integer primary key, val text);`) .run(); await create.txn?.wait(); const [tableName] = create.txn?.names ?? []; // Upload data via w3s // Convert the string to a Blob const content = "Hello world"; const blob = new Blob([content], { type: "text/plain" }); // Upload the data const cid = await client.uploadFile(blob);

This will provide us with a CID. As with the local pinning example above, we can insert that raw CID into the table and store the reference for later retrieval. Be sure to convert the CID object returned from w3s to a string upon inserting the data:

const sql = `insert into ${tableName} (val) values ('${cid.toString()}');`; const { meta: insert } = await db.prepare(sql).all(); await insert.txn?.wait();

Then, you can fetch the table data:

const { results } = await db.prepare(`select * from ${tableName}`).all(); console.log(results);

And the data will contain the CID:

[ { id: 1, val: "bafybeicrrabdtihvcyjppd4posgj7cbk3wr7xvokda7crxjt3wpul75ary" } ]

That's it! Tableland plus w3s is a great way to manage production scale upload large amounts of data while storing the references within the table itself.

Studio CLI gives a cost estimation for commands that will submit transactions

by Joe Wagner

The Studio CLI is a command line tool that allows folks to interact with the Studio via a shell. It's under active development, and new features are coming out regularly.

Next week you can expect to see an improvement to the confirmation messaging shown to anyone who is using the CLI to submit transactions, i.e. deploying a table and mutating table data. The new messaging uses the Ethers.js estimateGas method combined with the getFeeData method to give the user an estimate of how much running the command will cost. This information should act as a sanity check and ensure folk don't accidentally use more funds than they are expecting for a given command.

Here's a screengrab of the new messaging when you import a csv into a table

If you haven't checked out the CLI you can look at it on the npm site here: https://www.npmjs.com/package/@tableland/studio-cli or install it with npm via, npm install @tableland/studio-cli

If you have been using it and have any issues, comments, or suggestions get ahold of the core team on our Discord server!

Design & tensions

by Jim Kosem

At Textile, designing how we design and develop is an ongoing process, which is how it should be. Just as we iterate and refine software, we do the same with how we work, constantly tweaking and adjusting how the parts of the organisation move with and complement one another. Part of this involves writing.

We work completely remote and largely asynchronously, and therefore need to document well and often. Also because of this we need to approach tensions and how to best solve them in a different way. We write out tension documents to explain tensions we might feel and then at least one if not many proposal documents, with owners and timelines, which present options as to how to address them. The result is that often, what we design and develop starts with writing which forces us to think things through clearly and take time doing so rather than call endless meeting without unambiguous resolutions.


Other updates this week

  • We’re amidst our planning for the next few months—and a bit heads down to make sure we’re prepared for the coming months!

End transmission…

Want to dive deeper, ask questions, or just nerd out with us? Jump into our Telegram or Discord—including weekly research office hours or developer office hours. And if you’d like to discuss any of these topics in more detail, comment on the issue over in GitHub!

Textile Blog & Newsletter logo
Subscribe to Textile Blog & Newsletter and never miss a post.
  • Loading comments...