This is a frequently asked question, why use Twitter for identity in nodeStorage?

The TL;DR answer: It's easy for users, easier for me as a developer, and it might lead to collaboration with Twitter, which is something I'd like to see.

Now the more detailed answers.

1. We have to use something for identity. The choices were: cook up our own identity system, or use Twitter, Google, Facebook, LinkedIn, Amazon, GitHub, or...?

2. Cooking our own wasn't an option, at least not at this stage. An identity system is a complicated piece of software. Reliability is a huge factor. Building an identity system to scale is not a small undertaking. By using Twitter, I was able to focus on the other parts of nodeStorage and treat identity as a solved problem.

3. I know there is OpenID and IndieAuth. There are probably other ideas out there that I'm less familiar with. These approaches make it more complex for the user, they introduce hurdles. None of this matters if there is no user adoption, and even with a really easy identity system it's hard to get people to try it. I didn't want to make identity a research project, I needed something that "just worked" for users.

4. I already had code that implemented a Twitter-based identity system, and years of experience working with the Twitter API. Of the proprietary solutions it made the most sense, because I would have to start from scratch with any of the others.

5. The most important reason to use a proprietary identity system is user comfort. As a user myself I'm more likely to use my Twitter identity with another net app, than create another login that I have to manage. OAuth, which Twitter's identity is based on, is more complicated for me to implement as a developer, but I appreciate it as a user. I can easily turn off apps that I'm not using without having to change a password.

6. What if Twitter goes down? Seems not likely these days. Twitter seems very stable and performs really well. A more likely scenario is that Twitter might decide they don't want us using their system this way. Of course that's a concern, it could happen. If it does, we'll have to decide what to do then. If there's enough momentum behind nodeStorage, perhaps we can adapt to use a different identity system? Or come up with a driver structure that allows individual sysops to decide what identity they want to use? But if we haven't attained momentum, it seems unlikely that Twitter would care.

7. We're not sharing much data with Twitter. They know how much time each user spends with a nodeStorage-based app, but beyond that, they aren't getting any information. The user's data is stored elsewhere. (On the other hand I think it would be incredible if Twitter or one of the other system vendors would allow app developers to store per-user data on their servers. It would quickly establish them as the leading Internet application developers platform.)

8. Of all the potential identity providers, Twitter has the greatest reason to collaborate with independent developers. It would be hard to get Facebook's attention for a project like this. Embracing nodeStorage would be a huge win for both Twitter and open Internet developers. Because it defines an API for identity, it could be the basis for a new kind of net app, without lock-in. You could always add a new implementation behind the API if it became disadvantageous or impossible to use Twitter. Which would give Twitter an incentive to be fair to developers.

Here's a checklist for installing nodeStorage on Ubuntu.

Install Node.js

sudo apt-get update

sudo apt-get install nodejs

sudo apt-get install npm

sudo apt-get install nodejs-legacy

We also install npm, a requirement to run Node apps.

nodejs-legacy makes it possible to run apps by saying node app.js instead of having to use nodejs, an oddity of Ubuntu.

Install git

sudo apt-get install git

I like to install git, because it makes it easy to install nodeStorage from GitHub.

Download nodeStorage

git clone https://github.com/scripting/nodestorage.git

Install it

Change into the nodestorage directory you created in the previous step.

cd nodestorage

npm install

Create config.json

Launch your favorite Unix editor. I like nano because I'm a newbie, and it has a nice menu at the bottom of the screen if you don't know the commands.

nano config.json

Here's a template for config.json, copy the text, and paste it into nano in your terminal window.

Now I'm going to go through all the elements step by step, explaining what you have to do to set their values.

1. enabled -- this one is easy, leave it at true.

2. myPort -- enter the number of the port you want to use for nodeStorage. Make sure that this port is open in your firewall, if you have one on this server.

3. myDomain -- enter a domain name that points to this server. If you don't have one, you can enter the IP address of the machine, it will work as well as the domain name (but it's not mnemonic, and it's hard-coded, so if you move your installation, you'll have to change your apps).

4. where -- leave it as-is. It will store the public and private files in sub-folders of the nodestorage folder.

5. basePublicUrl -- take the domain and port and use them to form a URL. Be careful not to put a trailing slash on the URL as shown in the example. It must be exactly the form specified above.

6. twitterConsumerKey and twitterConsumerSecret -- I'll explain how to set this up in the next section. For now, leave them as set in the example. The strings are nonsense, just placeholders.

Save the file by typing Control-O, then exit with Control-X.

Launch nodeStorage

node storage.js

Set up your Twitter app

1. Go to apps.twitter.com and click Create New App in the upper-right corner. A page with a form appears, asking for details of your app.

2. You're probably configuring an app like MyWord Editor, but you have to call it something different. If it's for your book club, you could call it The Illium Book Club Blogs. If the name is already being used by someone, choose a different name. For my example I used "My test editor 2789". It was available.

3. The website URL will depend on whether you're hosting the app or someone else is. Use the URL of the app that's going to talk to Twitter. Since I am just using MyWord Editor I just entered the URL for the app on myword.io.

4. The callback URL for nodeStorage instances is the url of the app, as specified by basePublicUrl in config.json, followed by "/callbackFromTwitter" (don't include the quotes).

As an example, here's a screen shot of the page I used to set up Little Pork Chop, an app that uses nodeStorage.

1. If it all worked, you should have a Twitter app set up. Click on the Test OAuth button in the upper-right corner of the page, and it will show you two values, consumer key and consumer secret.

2. In the terminal, press Control-C to quit nodeStorage.

3. Open config.json in your editor, and replace the placeholder values for twitterConsumerKey and twitterConsumerSecret with these values and save the file.

Test your setup

1. Go to MyWord Editor.

2. Choose Switch Server in the Editor menu.

3. Enter the URL you entered for basePublicUrl above.

4. Sign on with the command in the rightmost menu.

5. You should get the Twitter authorization page. Authorize your app.

6. Enter a title, description and body.

7. Choose View Story in the Publish menu. You should see the story.

8. Look at your terminal. You should see various items in the log, showing MyWord Editor talking to your server.

9. Pat yourself on the back. You are now a DevOps dudess or duderino. On your way to being a Full Stack Developer.

Previous versions of nodeStorage could only store data in Amazon S3.

Starting with v0.77, we can also store data in the local file system.

Example

Here's an example of a config.json that tells nodeStorage to use the local file system.

What's different

1. There is a new where element, that specifies where the files are stored.

2. Because we specify a where element, s3Path and s3PrivatePath would be ignored, so we left them out. Important, they are still very much supported, so nodeStorage installations that predate v0.77 will continue to work with this version and future versions.

3. There's a new optional element, basePublicUrl, that says what all URLs should be prefixed with. If it's not present, we just add an http in front of the publicPath value to form the URL. This generally works if you're storing stuff in S3. It's still nice to have the ability to specify the URL,

This configuration works, I used it to test the development version of nodeStorage v0.77.

New feature in v0.77

We now can serve the public content managed my nodeStorage directly from the nodeStorage server. So if you're using the file system to store your data, you don't also have to run a web server to serve the content, nodeStorage will do it for you.

Of course you can use another server if you want. Just specify config.where.publicPath to point to a directory that's accessible to your web server, and set basePublicUrl so that it points to the location you're storing to when accessed over HTTP.

If you want to restrict access to your nodeStorage server to a small group of people, you can create a file containing the Twitter screen-names of the whitelisted users, and tell your nodeStorage server about it. This howto explains what to do.

1. The whitelist is a JSON file that contains the names of the whitelisted users. Here's an example. It must be in a place where your nodeStorage server can access it over HTTP.

2. Set up an environment variable called urlUserWhitelist. Its value is the address of the JSON file.

3. Be careful that the JSON is syntactically valid. Test it with jsonlint if you're not sure.

4. The nodeStorage server reads the whitelist once a minute. You don't have to restart the server to make your changes effective. Just wait a minute.

5. If you've established a whitelist, and a user is not on it, they will not be able to use the server to store data.

Here's the basic flow for connecting with the Storage server.

1. When your app starts up, call twGetOauthParams. You'll see how this is connected to the flow in step 4 below. That's the nature of OAuth, you have to understand how it flows before getting started.

2. Call twIsTwitterConnected to see if the user is currently logged into Twitter in your app. If it isn't put up a message saying they have to be logged into Twitter to use your app and provide a way for them to connect (see step 4).

3. If the user is logged into Twitter, get your prefs, open a document, do whatever you want with the storage. You can now make requests that get both public and private info for the user.

4. Suppose you have a button that says Log In To Twitter. When clicked, that button would call twConnectToTwitter, which calls the server which then begins the dance with Twitter. It sends the address of your app to the server, so it can relaunch it after the user authorizes. When that happens your app will pick up the OAuth credentials in step 1 above. Those credentials are kept in localStorage on the user's machine and are parameters to all the API calls.

twUploadFile

Call this API routine to upload a file to the server.

twUploadFile ("hello.txt", "hello world", "text/plain", false)

The last param, false, says that the file is not private.

There's a fifth, optional param, that's a callback that gets a bunch of info about the file, including its URL, if it's public.

Demo app

BTW, I'm just including this here to give you an idea of how this works. There's a simple demo app that illustrates it with working JS code.

I just made the nodeStorage repository public.

https://github.com/scripting/nodeStorage

A few notes

1. Please do not publicize this, or discuss outside of the server-snacks list at this time.

2. This is for developers. Later, it will make sense for people to run their own servers even if they're not developing apps using nodeStorage, but we're not there yet.

3. If you have questions, post them on the list. Be sure to click on all the links in the readme, and spend enough time scratching your head. Please RTFM. Thank you.

4. Marco Fabbri and Andrew Shell have been enormously helpful in getting this release ready. They made the project work, and made it a lot more fun for me. Many many thanks!

5. Can't wait to see what people do with nodeStorage.

nodeStorage is an open source project that brings simple S3-based storage to applications written in JavaScript that run in the browser.

For example, you could do a nice text editor or spreadsheet with nodeStorage as the back-end.

I built this server software to run behind apps like Radio3, Little Card Editor, Happy Friends and Little Pork Chop. Exactly the same software runs behind all these apps, which gives you an idea how versatile it is.

All web apps need storage

All web apps need to be able to store information on behalf of the user. We've had a couple of different approaches to work with that don't require an identity system -- cookies and local storage. These are good but the data is bound to a machine, not to a person. So if you went to a different machine, you'd have to start over.

Good for simple apps, but in order to get real utility, you need more.

Technologies

nodeStorage builds on three technologies: Node.js for the runtime, Twitter for identity and Amazon S3 for storage. The software could be ported to other runtimes, and other systems could be used for identity and storage.

To get things started we're using these technologies because they are well-deployed and reliable, and fit in well with the kinds of applications I wanted to make.

Philosophy

I like building software that runs in the browser, and has the smallest server footprint possible. It's a very efficient model, and gives the user maximum power.

Features

1. Public storage for publishing applications, private storage for preferences, personal documents.

2. Access to basic features of the Twitter API, sending and getting tweets, retweet, getting info about users, tweets, logging in and out.

3. Long-polling for realtime applications.

4. Whitelisting keeps access to an app limited to specific accounts.

5. Display of embedded tweets.

6. URL-shortening through the Bitly API.

7. Full source to an example app to help you get started coding.

Getting started

As of late January 2015, we've opened the software up to the server-snacks mail list. You'll find all the source code on the GitHub repository. There's a simple demo program running at macwrite.org. It's a text editor, for which we provide full source code. It's meant to show a curious developer how to build an app that runs on the other side of the Nodestorage API.

Where we go from here

I hope people kick the tires, read the code, ask questions. We'll keep polishing the code and improving the sample app, and building more real apps on top of the API. Anyone who does the same will be compatible. I hope that as a result, over time, it becomes a community, and a platform, and can evolve to the benefit of all developers who build on it and people who use their products.

Dave Winer