Weeknotes: October 30, 2023

A weekly digest of progress, updates, and insights from Tableland.

Begin transmission…

Postgres Logical Replication in Rust

by Avichal Pandey

Basin uses Postgres' streaming replication protocol to read the changes on a database and archive them in Filecoin. The basin cli is written in Go. It leverages mature libraries, especially pgx and pglogrepl. The latter offers higher-level abstractions to communicate with a Postgres server to extract WAL updates.

While trying to write a streaming replication client in Rust, I found that the logical replication libraries are not as mature. The rust-postgres project has a couple of open pull requests. These will add convenient higher-level methods for working with logical replication, but these still need to be merged.

Materialize's fork of rust-postgres has these features merged. It is stable and used by MaterializedInc in production. However, it has minimal documentation. The logical decoding example by @seddonm1 is an excellent starting point for someone trying to build apps that use Postgres streaming replication capabilities. It uses decoderbuf to extract the WAL messages.

I want to share a repository that uses Materialize's fork of rust-postgres mentioned above. It uses the wal2json plugin to extract and read WAL messages from Postgres and replicate them on an embedded DuckDB instance. The code illustrates the following:

  1. creating publications

  2. creating and using replication slots

  3. extracting JSON-encoded WAL messages

  4. sending timely commit and keepalive messages to the Postgres server

  5. using an embedded duck db instance to replicate changes

Wrapping up ETHOnline

by Dan Buchholz

ETHOnline 2023 is in the books! There were a lot of teams that built on Tableland…77 total, in fact. The judging process took quite some time, and although we wanted to reward everyone for their efforts, we ended up distributing 4 "top" prizes for first, second, and two runner up bounties, and then an additional 20 teams also split the pool. We also had a small pool for the teams that built with the (experimental) Basin CLI.

Here'a quick overview of the top projects:

Now, we're off to the Dataverse hackathon, which kicks off on November 9th and runs for about a month. See y'all there!

Progressing the Studio with more user research

by Jim Kosem

We've concluded our first round trip research, which means we began with an audit of the Studio software as it was, and then designed out based on assumptions of what would work for users, tested that, revised the designs based on those testing insights and then tested yet again. While following to an extent the well known Double Diamond design process, we have made our first massive step in creating evidence and insight based, user-centred design at Textile. As with every user-centred design effort, the process is always a negotiation, but we succeeded in validating some assumptions and future product developments like history and data inspector, while realising some UI assumptions weren't useful for people.

Optimism Goerli fixes in the SDK

by Dan Buchholz

During ETHOnline, we uncovered a bug in the SDK that affected downstream usage in clients like the Studio and even the CLI tool. Basically, Tableland has a devnet / "staging" environment for the Optism Goerli testnet, which is only used internally for testing purposes on a live network. Then, there's an Optimism Goerli testnet deployment that is open for anyone to use as they would any other chain that Tableland supports.

The problem was that the SDK didn't properly filter out the "staging" environment, so SDK users had no way to specify the correct Optimism Goerli chain. For example, if you used the Studio to deploy a table to Optimism Goerli, it created the table on the staging contract, and subsequent SDK APIs don't treat that data the same because it's materialized / exposed at a custom staging gateway.

Anyways, the change is now fixed with @tableland/[email protected], the Tableland CLI has this change incorporated, and the Studio will be releasing broader changes the include fix in the near term.

Other updates this week

  • We’re heads down on a lot from the protocol perspective, so keep an eye out over the coming months for more information about Tableland Basin and how it fits into the overall protocol vision. But, if you’re interested, feel free to reach out to [email protected] and drop a line about what you’re building…DePINs, especially!

End transmission…

Want to dive deeper, ask questions, or just nerd out with us? Jump into our weekly research office hours. Or, for hands-on technical support, you can join our weekly developer office hours.

And if you’d like to discuss any of these topics in more detail, comment on the issue over in GitHub!

Textile Blog & Newsletter logo
Subscribe to Textile Blog & Newsletter and never miss a post.
  • Loading comments...