Over 300 subscribers
New Tableland POST /query support
by Andrew Hill
Synthetic data is one of the most under-appreciated advances from current generative AI, poised to change data strategies across all industries, including web3. Gartner estimates that synthetic data will overshadow real data in AI models by 2030. That estimate is far too conservative and narrow in scope. The wave of synthetic data is coming. Whether it's enhancing privacy, accelerating research, democratizing access to high-quality data, or being used for malicious intent, the implications are vast and varied.
Despite its growing importance within AI, the conversation around synthetic data within the web3 community remains sparse. I've written up some of my thoughts on how synthetic can be used and improved by web3 technologies.
Read it here: https://mirror.xyz/tableland.eth/1atXBmvGRQPHqWnu6xlTkcGn1qo8LSFVp9beRMRDMTE
POST /queryby Bruno Calza
A new Tableland version has been released that includes the support of the HTTP method POST at the /query endpoint.
Now you can execute read queries without the need to URL encode them and send them inside the body.
Here's an example of querying the table healthbot_420_1:
curl -v --json '{"statement": "SELECT * FROM healthbot_420_1"}' https://testnets.tableland.network/api/v1/query
You can also provide formatting options:
curl -v --json '{"statement": "SELECT * FROM healthbot_420_1", "unwrap": true, "extract": true}' https://testnets.tableland.network/api/v1/query
Check out the Tableland docs for more details: here
by Dan Buchholz
We launched a new Discord channel that posts events for SQL logs, called #sql-logs in our server. It's pretty simple:
Runs a cron job every 15 minutes, querying a Tableland validator's system_evm_blocks and system_evm_events tables (internal validator tables that store onchain updates/log info).
Stores the run information in a local SQLite database to track the latest chains and blocks. Thus, as SQL logs/events are posted to Discord, the next run will know what the previous run included and start from that block number when searching for new logs.
Creates Discord embeds for all new SQL logs, including the SQL statement, any potential SQL syntax errors, and various links (/receipt, /tables, and /query APIs).
A few examples for creating tables, writing data, and a SQL syntax error.
Check out the source code here: https://github.com/tablelandnetwork/discord-sql-logs
The discord.js library makes interacting with Discord straightforward. To set up a connection, you can start with the following code, but first, you'll need to set up three environment variables:
DISCORD_BOT_TOKEN: The Discord bot token comes from creating a bot in the Discord Developer Portal at https://discord.com/developers/applications and getting a token under the Bot tab and Reset Token button.
DISCORD_WEBHOOK_ID, DISCORD_WEBHOOK_TOKEN: These webhook variables come from the server's URL of the webhook, which you create in the Discord channel settings: https://discord.com/api/webhooks/DISCORD_WEBHOOK_ID/DISCORD_WEBHOOK_TOKEN
import Database from "better-sqlite3"; import { Client, Events, GatewayIntentBits } from "discord.js"; import dotenv from "dotenv"; // Set up Discord client and ensure env vars are set up const client = new Client({ intents: [GatewayIntentBits.Guilds] }); const { DISCORD_WEBHOOK_ID, DISCORD_WEBHOOK_TOKEN, DISCORD_BOT_TOKEN } = process.env; if ( DISCORD_WEBHOOK_ID == null || DISCORD_WEBHOOK_TOKEN == null || DISCORD_BOT_TOKEN == null ) { throw new Error( "DISCORD_WEBHOOK_ID, DISCORD_WEBHOOK_TOKEN, or DISCORD_BOT_TOKEN is not defined" ); } // Code to connect & post to webhook... client.once(Events.ClientReady, async () => { // ... }); // Login to Discord and destroy the client after running the app await client.login(DISCORD_BOT_TOKEN); await client.destroy();
As noted, the app is structured to run on a cron schedule every 15 minutes and batch/post all updates seen since the last run. Part of this rationale is because, well, it's easy and free to do with GitHub Actions. If we take a look at our .github/actions/webhook.yml file, you can see that it simply runs our application code and then commits/stores the run information in the SQLite data/stat.db file.
name: SQL logs webhook on: workflow_dispatch: schedule: - cron: "*/15 * * * *" # Runs every 15 minutes permissions: contents: write jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Cache node modules id: cache-npm uses: actions/cache@v3 env: cache-name: cache-node-modules with: path: ~/.npm key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }} restore-keys: | ${{ runner.os }}-build-${{ env.cache-name }}- ${{ runner.os }}-build- ${{ runner.os }}- - name: Set up Node.js uses: actions/setup-node@v4 with: node-version: 20 - name: Install dependencies run: npm install - name: Build app run: | npm run build - name: Run app & post SQL logs to Discord run: | npm run start env: DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_BOT_TOKEN }} DISCORD_WEBHOOK_ID: ${{ secrets.DISCORD_WEBHOOK_ID }} DISCORD_WEBHOOK_TOKEN: ${{ secrets.DISCORD_WEBHOOK_TOKEN }} - name: Commit changes uses: stefanzweifel/git-auto-commit-action@v4 with: commit_message: "chore: new sql logs → update db state" file_pattern: "data/state.db"
If you review the tbl.ts file, you'll find two queries made to the Tableland network:
Query for latest system blocks:
SELECT chain_id, max(block_number) as block_number, timestamp FROM system_evm_blocks GROUP BY chain_id;
Query for SQL events/logs for each chain and a specified block range—where the ${…} are string templated variables injected from a combination of the query above and the local SQLite database’s state:
SELECT chain_id, block_number, tx_hash, event_type, json_extract(event_json,'$.Caller') as caller, json_extract(event_json,'$.TableId') as table_id, json_extract(event_json,'$.Statement') as statement FROM system_evm_events WHERE block_number > ${range.prev_block_number} AND block_number <= ${range.block_number} AND chain_id = ${range.chain_id} AND (event_type = 'ContractCreateTable' OR event_type = 'ContractRunSQL') ORDER BY block_number ASC;
Thus, every cron will fetch the state and create Discord embeds with this new data.
In our clients.once(Events.ClientReady, ...) code, we start by running the logic described to query Tableland for the latest state and structure our data. Then, we can call the fetchWebhook method to connect and post to the webhook:
// Fetch the Discord webhook and send the SQL logs as embeds const webhook = await client.fetchWebhook( DISCORD_WEBHOOK_ID, DISCORD_WEBHOOK_TOKEN ); if (webhook == null) { throw new Error("No webhook found"); } const embeds = buildDiscordEmbeds(sqlLogs); await sendEventsToWebhook(webhook, embeds);
The buildDiscordEmbeds and sendEventsToWebhook are custom methods that do the following:
Create Discord embeds that contain the SQL log information (the embed fields can be found in the embed.ts file):
const embedFields = [{ name: "...", value: "..." }]; const embed = new EmbedBuilder() .setTitle("New SQL Logs") .setColor(0x815691) .setFields(embedFields) .setTimestamp(new Date()) .setFooter({ text: "❤️ SQL Logs Bot", iconURL: "https://bafkreihrg4iddyor2ei6mxxdy6hqnjsmquzcnllvoqndfb636i5s4yinma.ipfs.nftstorage.link/", });
Post the embeds to the webhook:
await webhook.send({ username: "SQL Logs Bot", avatarURL: "https://bafybeiezqhnetm6iidwpkpcmaoczjvlldxauffs5566t5sungxqabgtm7q.ipfs.nftstorage.link/", embeds: [embed], });
And that's it! If you want to see what the full implementation looks like, you can review the source code noted above. Plus, it's set up to be able to work in your own Discord, too—you just need to set up your environment variables accordingly!
by Marla Natoli
GEODNET is gathering geospatial data using space weather stations to create a user-powered RTK network (the application of surveying to correct common errors in current satellite navigation (GNSS) systems)*. GEODNET claims to enable the measurement of Space Weather via a dense set of measurements at a radically lower cost using blockchain technology and a decentralized structure. This vision for this data is for it to be combined with other advances in sensor technology and AI for use cases like autonomous driving, robotics, AR, and more.
At Textile, we believe that the vast amounts of potential data created by organizations like GEODNET pose an incredible opportunity, especially when combined with other data sources. This opportunity guides our recent work around a decentralized object storage solution (codenamed Basin), which enables new use cases and unlocks value for data created outside of traditional walled gardens. Here are some use cases for Basin that are relevant to GEODNET:
Simplify multi-writer workflows by adding verifiability to data coming from GEODNET devices, establishing provenance to make data traceable to its origin (a specific GEODNET device or GEODNET as an organization)
Combine data from GEODNET with other data sources, allowing many actors to write to a single data pool with programmable read/write access control. (Imagine simplifying the ability to pool and use data from several DePINs like GEODNET, Hivemapper, Dimo, WeatherXM, etc., and rewarding the network participants who created that data fairly and transparently).
Provision access & monetize ****data with programmable read & write access control, configurable pricing & licensing, and flexible governance options
Bring transparency to rewards calculations and/or proofs with Basin + a compute solution, having access to data both in near real-time to calculate rewards—and over the long term, to handle dispute resolutions
If you’re a DePIN that’s currently thinking about how to decentralize your data infrastructure, add transparency, or facilitate the use of the data your network creates, we’d love to hear from you. Set up some time here or join our Discord and get in touch.
Sources: https://en.wikipedia.org/wiki/Real-time_kinematic_positioning; https://geodnet.com/index.html
ICYMI—we published a detailed blog on the events we’ll be hosting and attending at ETHDenver 2024: here. TL;DR:
February 27th: DePIN Day
February 28th: R3al World; DeSci Summit
February 29th: FIL Dev Summit; Devtopia
March 1st: Proof of Data Summit (the event we’re hosting!); Filecoin Orbit Happy Hour
Want to dive deeper, ask questions, or just nerd out with us? Jump into our Telegram or Discord—including weekly research office hours or developer office hours. And if you’d like to discuss any of these topics in more detail, comment on the issue over in GitHub!
Are you enjoying Weeknotes? We’d love your feedback—if you fill out a quick survey, we’ll be sure to reach out directly with community initiatives in the future!: Fill out the form here
Collect this post as an NFT.
