Integrating Powergate: Intro to the Powergate JavaScript Client
Textile built Powergate as a way to bridge the gap between IPFS and Filecoin storage. You get flexible data storage configurations optimized for long term, cryptographically verified, affordable storage on Filecoin, and high availability, fast, distributed storage on IPFS, or both simultaneously. According to the Powergate docs:
Powergate is a multi-tiered file storage API built on Filecoin and IPFS, and and index builder for Filecoin data. It's designed to be modular and extensible.
Powergate is meant to be integrated into other systems and applications. It's the easiest way to leverage IPFS and Filecoin, and by using Powergate to orchestrate how they work together, you get the best of both worlds plus the added smarts of Powergate's powerful data storage configuration mechanism.
If you'd like more information about Powergate in general, please see the repo's README and FFS design documentation.
Options for Integration
Today, there are a few tools we provide for integrating Powergate into your system or application. If you're building in Go, there's the Go Powergate client. If you're writing shell scripts or want to quickly experiment with Powergate APIs, the CLI would be a great tool (Installable executables coming soon -- for now, follow instructions in the README about how to install from source). The newest addition is our TypeScript/JavaScript Powergate client, and that's what we'll focus on in this post.
As a side note, all of the clients I just mentioned are built using gRPC services exposed by Powergate, so in theory any type of client could be built in any language supported by gRPC. Browse around the repo to see the various .proto
files defining these services. For example, the service definition for the Deals module can be found here.
The Example Project

The project I'll work through in this post is a simple Node.js web app built with Express as a server, router, and middleware and it uses Pug templates for rendering HTML. You can find the finished product in the examples folder of the Powergate JavaScript client repo. I'll use TypeScript because I like it, but you don't have to. The app will display some basic information about the Powergate instance it's connected to and allow users to authenticate using GitHub OAuth. Once authenticated, the web app will display additional information about the user's own FFS instance.
Initial Project Setup
If you have an existing project you'd like to use as we walk through this example, you'll just need to install @textile/powergate-client
and then adapt further code examples I provide to whatever tools your project uses.
npm install @textile/powergate-client
For everyone else, I created a minimal application you can clone or fork that includes all the dependencies we'll need, plus some basic plumbing for user authentication, and UI. To get started with it, run the following:
git clone https://github.com/textileio/node-starter.git
cd node-starter
npm install
Setup a GitHub OAuth App
To get GitHub authentication working in the example app, you'll need to log into your GitHub account and create a new OAuth App. You'll find this under Settings > Developer Settings > OAuth Apps. Create a new one, you can use http://127.0.0.1:3000
as the Homepage URL and http://127.0.0.1:3000/auth/github/callback
as the Authorization callback URL. Take note of the generated client id and secret provided after you submit the form.
Back in our project, copy .env.example
to .env
and open it for editing. Update GITHUB_CLIENT
and GITHUB_CLIENT_SECRET
with the values from your new OAuth app.
Important: Don't check your .env
file into any public repository since it contains application secrets. The .gitignore
in our starter project is already set up to do the right thing here.
Initial Run
You should now be able to run the app in debug mode:
npm run debug
And you should see the default application home page at http://localhost:3000.

Application Anatomy
Our starter app already has some basics that we'll take advantage of. Here's an overview of the important pieces and how we'll use them when integrating Powergate.
src/server.ts
is the main entry point to our application.
src/util/env.ts
gives us access to variables defined in our .env
file. We'll use those to specify where to connect to our Powergate instance as well as access other application secrets.
src/config/passport.ts
configures our Passport GitHub authentication strategy. We'll allocate Powergate resources for authenticated users.
src/models/user.ts
contains our User
data model and provides SQLite-based persistence of user data. We'll extend this model to associate Powergate information with users.
src/app.ts
is where our Express application lives. All routes and route-handling middleware are defined here. We'll use middleware to interact with the Powergate client, querying for data to display and creating Powergate resources for authenticated users.
views/
contains Pug templates that we'll use to render information about Powergate and authenticated user Powergate resources.
Powergate currently uses two categories of API access; Unauthenticated requests to access general information about the Powergate and Lotus node health, network, miners and more, and authenticated requests to access to individual FFS instances. A single FFS instance provides partitioned access to and management of data storage on IPFS and Filecoin.
Our goal with this example app is to display some general information about Powergate using the unauthenticated APIs, and then create a FFS instance for each authenticated user. The same FFS instance should be used by a user that logs out and then returns to the app at a later time. We'll display a bit of information about the authenticated user's FFS instance.
Running Powergate
Of course, in order to integrate Powergate into our app, we'll need an instance of Powergate running. Textile is considering offering hosted Powergate services, so if that's something you'd be interested in, please get in touch. For now, there are a few options to get Powergate running yourself.
- BYO - Powergate depends on connecting to IPFS and Lotus nodes. You are free to run your own and then run the Powergate server
powd
yourself, configuring it with the addresses of your IPFS and Lotus nodes. - Docker + testnet - We provide a Docker compose configuration that will spin up Powergate, IPFS, and Lotus, plus some extra metrics tracking and visualization tools. The resulting setup runs on the Filecoin testnet (and mainnet once it launches). This is good for a production setup, but will feel slow for development purposes.
- Docker + localnet - We created a special configuration of Lotus that runs a filecoin network locally at high speed, and it works great for testing and development. It's easy to use via a localnet Docker compose configuration. This is how we'll be running Powergate and its dependencies for this example app.
The Powergate releases page includes bundled up Docker Compose files that use appropriate Docker image versions of Powergate, Louts, and IPFS. Let's check our package-lock.json
to see the appropriate release to download and run. Look for @textile/grpc-powergate-client
. Its version number corresponds to the version of Powergate the underlying gRPC bindings were created from.
"@textile/grpc-powergate-client": {
"version": "0.0.1-beta.13",
...
},
Download the powergate-docker-<version>.zip
from the appropriate release, in this case, version 0.0.1-beta.13
. Assuming you have Docker Desktop installed, start up Powergate using the provided Makefile
:
unzip powergate-docker-v0.0.1-beta.13.zip
cd powergate-docker-v0.0.1-beta.13
BIGSECTORS=true make localnet
Important note on BIGSECTORS: When running the localnet setup, the Lotus node is configured with a mocked sector builder, using either "small" or "big" sector sizes. The practical effects of this configuration are on the size of files you can store in the localnet and how quickly the storage deals will complete. Using BIGSECTORS=false
will limit you to storing files of around 700 bytes and deals will complete in 30-60 seconds. Using BIGSECTORS=true
will allow you to store files anywhere from 1Mb to 400Mb, but deals will complete in 3-4 minutes. Be sure to choose the value that makes sense for your development scenario.
Connecting to Powergate
First, we'll configure the project to connect to our Powergate instance and then create the client we use to make calls to Powergate. Open .env
for editing and add a new variable for the Powergate API host. The below value is correct if you're running our Docker compose setup, but adjust it as needed if you're running Powergate elsewhere:
POW_HOST=http://0.0.0.0:6002
Then, let's add an the exported value of this variable to src/util/env.ts
so we can easily read it's value from elsewhere in the application:
export const POW_HOST = process.env["POW_HOST"]
Now we can create our instance of the Powergate client, connecting to the configured host. In src/app.ts
we'll import our POW_HOST
variable and the Powergate client factory function createPow
, and finally create our Powergate client we'll call pow
:
import { createPow } from "@textile/powergate-client"
import { EXPRESS_PORT, POW_HOST, SESSION_SECRET } from "./util/env"
const pow = createPow({ host: POW_HOST })
Unauthenticated Requests + Display
Now that our Powergate client is ready to go, we can use it to query some data from Powergate, and then we'll display that data on the home page of the web app. In src/app.ts
, we'll update our /
route handler to call some Powergate APIs and pass the resulting data to the render
function so it's available in our HTML template. Most Powergate functions return Promise
s since they are making async calls over a network. For that reason, we also need to label our route handler function as async
, use try/catch
as usual with Promise
s, and call the handler's next
parameter in case of an error. Here's the updated version:
app.get("/", async (_, res, next) => {
try {
const [respPeers, respAddr, respHealth, respMiners] = await Promise.all([
pow.net.peers(),
pow.net.listenAddr(),
pow.health.check(),
pow.miners.get(),
])
res.render("home", {
title: "Home",
peers: respPeers.peersList,
listenAddr: respAddr.addrInfo,
health: respHealth,
miners: respMiners.index,
})
} catch (e) {
next(e)
}
})
We've now passed the results from our calls for peers, listen address, health status, and Filecoin miners into our render function, so we can update views/home.pug
to render that information. I updated the header text, and to avoid needing to make more UI, I'm just stringify
ing the data and printing it inside <pre>
tags:
extends layout
block content
h1 Node Info
p.lead A sample of generally available data, no auth required.
hr
.row
- const jsonPeers = JSON.stringify(peers, null, 4)
- const jsonAddr = JSON.stringify(listenAddr, null, 4)
- const jsonHealth = JSON.stringify(health, null, 4)
- const jsonMiners = JSON.stringify(miners, null, 4)
.col-sm-6
h2 Node Health
pre= jsonHealth
.col-sm-6
h2 Listen Address
pre= jsonAddr
.col-sm-6
h2 Peers
pre= jsonPeers
.col-sm-6
h2 Miners
pre= jsonMiners
Refresh, and you should see:

User Model Update
We need to update our User
model to represent the FFS instance associated with each user. We'll add a ffsToken?: string
property to the User
type in src/models/user.ts
. It will hold the token returned when we create a FFS instance for a user:
export type User = {
gitHubId: string
email: string
ffsToken?: string
}
Next, update the User
references and SQL statements in src/models/user.ts
for the User
CRUD operations, taking into account the new ffsToken
property. For example, findByGitHubId
becomes (notice we update the SQL statement with a new field and the response object with a new field):
export const findByGithubId = async function (gitHubId: string): Promise<User | undefined> {
const q = "SELECT gitHubId, email, ffsToken FROM users WHERE gitHubId = ?"
const row = await db.get(q, gitHubId)
if (!row) {
return undefined
} else {
const user: User = {
gitHubId: row.gitHubId,
email: row.email,
ffsToken: row.ffsToken,
}
return user
}
}
This is a little tedious, so you may want to just copy the final version from the example app repo.
FFS Creation Middleware
In src/app.ts
, the route /auth/github/callback
is called when GitHub redirects the user back to our application as part of the OAuth process. It currently finishes the authorization process and then redirects the user to /user
. We'll add another handler to that flow, after completing the authorization process, but before redirect, to create a FFS instance if needed for the now-authenticated user then update and save the User
instance:
import { save, User } from "./models/user"
app.get(
"/auth/github/callback",
passport.authenticate("github", { failureRedirect: "/" }),
async (req, _, next) => {
if (req.user) {
const user = req.user as User
if (user.ffsToken) {
pow.setToken(user.ffsToken)
return next()
} else {
try {
const createResp = await pow.ffs.create()
user.ffsToken = createResp.token
await save(user)
pow.setToken(user.ffsToken)
next()
} catch (e) {
next(e)
}
}
} else {
next(new Error("no user found in session"))
}
},
(_, res) => {
res.redirect("/user")
},
)
Our handler is async
since it calls the Powergate client's Promise
-based ffs.create()
and the User
save()
functions.
We first make sure we can access our Express user object (which we know is our User
type). If not, this is an error.
Then we check if the user already has a ffsToken
. If so, this is a returning user and we're done. We call next()
so the request handler chain simply continues.
If the user doesn't have a ffsToken
, we create a new FFS instance, set the user's ffsToken
property with the returned token and save the updated User
.
The last step is calling pow.setToken(user.ffsToken)
. This passes the user's FFS token into our Powergate client, and the client will now send that token along with each request to Powergate so it knows which FFS instance the user is interacting with.
Important note: The communication with Powergate is currently unencrypted http so the FFS token is sent in plain text. We plan on providing tooling to support encrypted https in an upcoming release.
Authenticated Request + Display
In src/app.ts
, we already have a route, /user
, thar requires authentication (enforced by the passportConfig.isAuthenticated
middleware). Having an authenticated user at this point implies that we've already created a FFS instance for that user, and that our pow
client is configured to use that FFS instance's token. So let's now make an authenticated call to Powergate. We'll do this by updating the handler for the /user
route:
app.get("/user", passportConfig.isAuthenticated, async (_, res, next) => {
try {
const info = await pow.ffs.info()
res.render("user", {
title: "User",
info: info.info,
})
} catch (e) {
next(e)
}
})
Here, we call ffs.info()
to which returns some information about the user's FFS instance. We pass the result into our render function so we can display it in the updated user.pug
template:
extends layout
block content
h1 FFS (#{info.id})
p.lead Data for your FFS instance.
hr
.row
- const jsonInfo = JSON.stringify(info, null, 4)
.col-sm-6
h2 Instance Info
pre= jsonInfo
If you refresh and make sure you're logged in, you should see:

If you log out and log back in, you should see that the FFS instance id remains the same, showing that the same user will always be interacting with their unique FFS instance.
Wrap Up
In this example, we've retrofitted a very simple, but typical, Node.js web app with Powergate capabilities. With this simple change, we can view information about our Powergate, IPFS, and Lotus nodes, and users now have access to powerful data storage on IPFS and Filecoin through their own FFS instance. I hope that seeing one way to map Powergate concepts onto an existing application gives you ideas about how you might integrate Powergate into your own systems.
Next Steps
The Powergate JavaScript client is brand new and improving every day. The most commonly used Powergate APIs are available already, but we'll be adding more and keeping the client up to date with the core Powergate APIs. Be sure to star or subscribe to the repo to stay updated.
Creating FFS instances for users is really just scratching the surface of what's possible with Powergate. FFS is the main API you'll be interacting with, and I'd encourage you to extend this example to use more of its functions for adding and retrieving data to and from IPFS and/or Filecoin. In fact, that may be a great follow up example app and blog post... Stay tuned!
Want to start building with Filecoin today? Want to use it with NodeJS or even from your application? How about IPFS integration? Check out this Introduction to the Powergate JavaScript Client: https://t.co/5TesRBj5AG #filecoin #ipfs #javascript
— Textile (@textileio) June 15, 2020