I’ve joked for over a year now that if there ever was a HTML 6 its marquee feature would be Node.js. In other words I should be able to write a packaged app that sits on a device that has one part running in a browser/webview and another part running a local Node.js instance that I can use to accept incoming request. Furthermore I need to be able to build and deploy HTML 6 packaged apps on at least (but not at most) - Android, iOS, Linux, OS/X, Windows desktop and Windows RT. The purpose of this article is to lay out my nefarious plan for making HTML 6 packaged apps real.
1 Disclaimer
This article contains my opinions, not my employers. You have been warned.
2 Groking the model or It’s the Web, Stupid!
The model I want is one where someone writing an application can write their UX in HTML 5 + Javascript + CSS and simultaneously launch an on-device listener using Node.js. The UX and Node code should be loosely coupled because they will have different life cycles. For example, when we use this infrastructure for Thali we will want the Node code to essentially always be running as a background service while the front end UX code only needs to run when the user actually needs it. We do need a secure way for the HTML 5/Javascript/CSS code to talk to the Node.js code but there is this HTTP thing I’ve heard about, I’d like to use that.
We then need a way to take these two loosely coupled components, the HTML 5/Javascript/CSS UX and the node.js local listener and package them together and deploy them as a unit on all of our supported platforms.
3 So what exists today?
Below are the efforts I know of today that seem relevant to making HTML 6 packaged apps real. If I missed something, please let me know!
Cordova In terms of packaged HTML 5 apps for mobile platforms Cordova seems to be a pretty big deal. But it doesn’t (currently) support Node.js and it doesn’t have a desktop story.
node-webkit This effort marries Chromium and Node.js to create a single run time environment that generates self contained apps, but only for the desktop. Their model makes the UX and node.js code much more tightly coupled than I am looking for. And there’s no mobile story I know of. But it’s still pretty cool. I need to grab some time to seriously play with it.
jxcore This is a fork of Node. Amongst other things they allow folks to write node apps and then package them for all the desktop environments but they have also said they will support iOS, Android and WinRT. Their desktop work is in beta and their mobile work is in alpha. They are closed source right now but have claimed that they will go open source before their final release.
tidesdk/TideKit tidesdk was an open source effort to create a single development environment for HTML5 packaged apps that would run on all the usual desktop OS’s. TideKit is apparently the replacement and is closed source in a closed beta. It claims to support writing packaged HTML 5 apps for, well, everything, e.g. both mobile and desktops. They do talk about Node.js but I believe they mean integrating with it when it’s running server side. But I could be wrong.
webinos webinos is a foundation that has built a Javascript based peer to peer environment for desktop OS’s as well as Android. They mention iOS but it’s not clear if they just mean they can produce server content that runs on iOS browsers or actually support iOS packaged apps. They have chosen to not use HTTP as their main protocol and to instead create their own JSON API sent over TLS sockets. But they do use node.js to enable their peer to peer functionality. I’m not at all happy that they don’t use HTTP as their main protocol and that they invented their own discovery protocol (why not just use mDNS or SSDP?) but our goals, especially in the context of Thali, are very close. So we should definitely talk. Thanks to Paddy Byers of Anode for pointing them out to me!
node.js They support all the desktop OS’s today and have put in support for Android (but see here for instructions to make it actually work). I’m not aware of any plans on their part to support iOS or WinRT. And the node folks aren’t directly worried (at least not that I’m aware of) about packaging themselves up into apps. But the Android work strongly implies that something is probably going to have to give there. We shall see.
neu.node This is an older, possibly no longer relevant, effort to create a Node like environment for iOS. It was started before Apple loosened their restrictions on interpreted code in iOS applications.
nodeapp This is an attempt to build a Node like environment for iOS but using Apple’s Javascript engine rather than V8.
Chrome Packaged Apps This is an effort by Google to deliver HTML 5 packaged apps everywhere including all the usual desktop OSs as well as (via Cordova and Crosswalk on Android) for mobile OS’s. For obvious reasons Chrome Apps focuses on making it easy to use Google’s cloud infrastructure and it doesn’t seem to have any kind of service model ala Node.js.
4 How I’d like to see things happen
In talking with folks I think it’s easier if I make my own biases clear. I can and do change my mind but at least by understanding where my thinking is starting hopefully those kind enough to spend time talking to me will find it easier to navigate me away from my delusions.
It seems to me that the ideal outcome would be:
-
Cordova adopts the desktop as a compilation target.
- I’d be happy to see this just be a wrapper around node-webkit but abstract away how node apps are started since it’s a non-goal to provide the level of integration node-webkit provides between the UX and Node environments. And yes I did see the project to combine node-webkit and Cordova but it seems moribund. Still, it could be a great place to start.
- Alternatively I’d be happy to just have us create a native app that opens up the user’s default browser and simultaneously starts a node.js instance in the background.
-
Node.js runs on iOS and WinRT (it’s already on Android)
- From a compatibility perspective the ideal solution here would be for Node.js to directly support iOS and WinRT which would mean getting V8 to run on those platforms. There is some hope for iOS. WinRT is a harder nut to crack, but I’ll have to see what can be done. The main issue, btw, is that WinRT doesn’t allow 3rd party code JITing.
- A substantially less attractive option is to create node ’like’ alternatives for those platforms. That is essentially what neu.node and nodeapp are doing. They are not Node but they try to be faithful to the Node APIs. This inevitably creates compatibility issues and in practice largely means that native add-ons are a no-go since those are very specific to V8. So native V8 support is certainly the desired direction.
-
Provide application life cycle hooks for both the HTML 5 front end and for Node.js on all supported platforms
- Cordova already has a model for this, we would need to also expose it to Node.js.
-
Create a Node.js Cordova plugin
- A developer would create their Cordova app as usual and include a Cordova plugin that can call out to a node app. We’ll have to plug into the Cordova build process and based on the target platform we would then run NPM, compile any native add-ons for the target platform and then package node, source files, etc. together.
This post is quite interesting!
I’m working for a while with Cordova (since 2009) but as you mention the desktop part is missing. In an other hand node-webkit is quite a fantastic approach but it does not cover the mobile part.
I would say, NodeJS+Webkit+NPM+Cordova can become a real full OS, with plenty of advantages.
– NodeJS: the service provider (specially network)
– WebKit: the UI provider
– NPM: the package provide
– Cordova: add the missing mobile events
And we need to add a thread API. Actually something more modern such as Grand Dispatch Central.
I’started to prototype something like that on a root android device (with a chroot Linux install for the moment)
Strictly speaking we have at least two different flavors of thread API already. One is the cluster model in node.js (which yes, is processes, not threads) and the other is threads-a-gogo which layers a thread like API over the isolate/process model. While I can see scenarios where both approaches are useful there is one particular scenario I’m interested in where both fail pretty spectacularly and that is for map/reduce.
Spawning multiple isolates and/or full processes just sucks because as far as I understand each isolate (and certainly each process) contains a full V8 implementation which eats up around 5 megs I believe of space. Well my phone has 4 cores, I’d like to use them all but not at a 20 meg overhead before I even get started!
But the even worse problem is memory sharing. Ideally one wants to be able to share results during a map reduce but using IPC or string copy is off the inefficiency scale.
I’m no V8 expert but I suspect there is a solution to this mess. It believe it is possible to share data structures between isolates in V8 but only if the data is created outside of the JS engine. To keep our sanity with regard to threading this sharing would need to be immutable. But trying to deal with immutability in any efficient way in Javascript is really going to suck.
But imagine if we took a language like Clojure and adapted it’s existing immutable parallel processing primitives? Clojure already compiles to Javascript. I think this could be an amazing way to write high performance shared memory cross thread code in node.js.
Just a thought.
I’m looking for such possibilities from years now and i tried so much things like building my webview for a lot of plateform. Cordova/phonegap seems to help because with plugins we can use some hardware and so on ( writing a file and reading it is the most important thing isn’t it). I hope tidekit is not “cordova make-up”… In fact, to sum up, we design a html+css+js site but we lack of a webview for everywhere and some links with the os system (like js and c language working together for example).
I think everyone understands that this has to end. That we have to reach the point where Cordova runs everywhere. Cordova has a great story on mobile but desktop is still weak. I was looking at NodeWebkit but I’ve heard good things about . I hope in the next few months to get to spend cycles on this part of the problem.
That last time I looked, node webkit was completely inaccessible, at least when using voiceover on mac.
Accessibility is a huge deal but I’m surprised to hear this is a problem since I assume that Chromium (that node-webkit is wrapping) is, I believe, compliant with the usual accessibility features. It may be that they haven’t properly hooked them up on the mac. We are also looking at atom-shell. Right now we are primarily focused on mobile, desktops come next. I’m pretty sure however that this should first and foremost be a Cordova feature that then gets mandatory exposure on all platforms. I added https://www.pivotaltracker.com/story/show/88623160 as a work item to start the process of identifying the right people who can give me a list of the accessibility requirements we need to meet and then go from there. Thank you for raising this.