For my internship at IBM, we're going to be doing a lot of work on
Node.js. This is awesome: Node is a great platform. However, I very quickly
discovered that the state of Emacs ↔ Node.js integration is dilapidated at best
(as far as I can tell, at least).
One of the first tools I came across was the swank-js / slime-js
combination. However, when I (after a bit of pain) got both setup, slime
promptly died when I tried to evaluate the no-op function: (function() {})().
Many pages describing how to work with Node in Emacs seem woefully out of
date. However, I did eventually find nodejs-repl via package.el. This
worked great right out of the box! However, it was missing what I consider a
killer feature: evaluating code straight from the buffer.
Buffer Evaluation: Harder than it Sounds
Most of the languages I use that have a REPL are Lisps, which makes choosing
what code to run in the REPL when I mash C-x C-e pretty
straightforward. The only notable exceptions are Python (which I haven't used
much outside of Django since I started using Emacs) and JavaScript (which I
haven't used an Emacs REPL for before). Thankfully, while the problem is
actually quite difficult, a collection of functions from js2-mode, which I
use for development, made it much easier.
The first thing I did was try to figure out how to evaluate things via
Emacs Lisp. Thus, I began with this simple function:
(defun nodejs-repl-eval-region (start end)
"Evaluate the region specified by `START' and `END'."
(let ((proc (get-process nodejs-repl-process-name)))
(comint-simple-send proc (buffer-substring-no-properties start end))))
It worked! Even better, it put the contents of the region in the REPL so that
it was clear exactly what had been evaluated! Whole-buffer evaluation was
similarly trivial:
(defun nodejs-repl-eval-buffer (&optional buffer)
"Evaluate the current buffer or the one given as `BUFFER'.
`BUFFER' should be a string or buffer."
(interactive)
(let ((buffer (or buffer (current-buffer))))
(with-current-buffer buffer
(nodejs-repl-eval-region (point-min) (point-max)))))
I knew I wasn't going to be happy with just region evaluation, though, so I
began hunting for a straightforward way to extract meaning from a js2-mode
buffer.
js2-mode: Mooz is my Savior
Mooz has implemented JavaScript parsing in Emacs Lisp for his extension
js2-mode. What this means is that I can use his tools to extract meaningful
and complete segments of code from a JS document intelligently. I
experimented for a while in an Emacs Lisp buffer. In short order, it became
clear that the fundamental unit I'd be working with was a node. Each node
is a segment of code not unlike symbols in a BNF. He's implemented many
different kinds of nodes, but the ones I'm mostly interested in are
statement and function nodes. My first stab at function evaluation looked
like this:
(defun nodejs-repl-eval-function ()
(interactive)
(let ((fn (js2-mode-function-at-point (point))))
(when fn
(let ((beg (js2-node-abs-pos fn))
(end (js2-node-abs-end fn)))
(nodejs-repl-eval-region beg end)))))
This worked surprisingly well! However, it only let me evaluate functions
that the point currently resided in. For that reason, I implemented a simple
reverse-searching function:
(defun nodejs-repl–find-current-or-prev-node (pos &optional include-comments)
"Locate the first node before `POS'. Return a node or nil.
If `INCLUDE-COMMENTS' is set to t, then comments are considered
valid nodes. This is stupid, don't do it."
(let ((node (js2-node-at-point pos (not include-comments))))
(if (or (null node)
(js2-ast-root-p node))
(unless (= 0 pos)
(nodejs-repl–find-current-or-prev-node (1- pos) include-comments))
node)))
This searches backwards one character at a time to find the closest
node. Note that it does not find the closest function node, only the
closest node. It'd be pretty straightforward to incorporate a predicate
function to make it match only functions or statements or what-have-you, but
I haven't felt the need for that yet.
My current implementation of function evaluation looks like this:
(defun nodejs-repl-eval-function ()
"Evaluate the current or previous function."
(interactive)
(let* ((fn-above-node (lambda (node)
(js2-mode-function-at-point (js2-node-abs-pos node))))
(fn (funcall fn-above-node
(nodejs-repl–find-current-or-prev-node
(point) (lambda (node)
(not (null (funcall fn-above-node node))))))))
(unless (null fn)
(nodejs-repl-eval-node fn))))
You Know What I Meant!
My next step was to implement statement evaluation, but I'll leave that off
of here for now. If you're really interested, you can check out the full source.
The final step in my rather short adventure through buffevaluation-land was a
*-dwim function. DWIM is Emacs shorthand for Do What I Mean. It's seen
throughout the environment in function names such as comment-dwim. Of
course, figuring out what the user means is not feasible – so we guess. The
heuristic I used for my function was pretty simple:
- If a region is active, evaluate it
- If the point is at the end of the line, try to evaluate the statement on
that line (works with multiline statements thanks to Mooz's awesome work)
- Otherwise, evaluate the first statement or function found
This is succinctly represent-able using cond:
(defun nodejs-repl-eval-dwim ()
"Heuristic evaluation of JS code in a NodeJS repl.
Evaluates the region, if active, or the first statement found at
or prior to the point.
If the point is at the end of a line, evaluation is done from one
character prior. In many cases, this will be a semicolon and will
change what is evaluated to the statement on the current line."
(interactive)
(cond
((use-region-p) (nodejs-repl-eval-region (region-beginning) (region-end)))
((= (line-end-position) (point)) (nodejs-repl-eval-first-stmt (1- (point))))
(t (nodejs-repl-eval-first-stmt (point)))))
The Beauty of the Emacs Development Process
This whole adventure took a bit less than 2 hours, all told. Keep in mind
that, while I consider myself a decent Emacs user, I am by no means an ELisp
hacker. Previously, the extent of my ELisp has been one-off advice functions
for my .emacs.d. Being a competent Lisper, using ELisp has always been pretty
straightforward, but I did not imagine that this project would end up being
so simple.
The whole reason it ended up being easy is because the structure of Emacs
makes it very easy to experiment with new functionality. The built-in Emacs
Lisp REPL had me speeding through iterations of my evaluation functions, and
the ability to jump to functions by name with a single key-chord was
invaluable. This would not have been possible if I had been unable to read
the context from the sources of comint-mode, nodejs-repl and
js2-mode. Even if I had just been forced to grep through the codebases
instead of being able to jump straight to functions, it would have taken
longer and been much less enjoyable.
The beautiful part of this process is really how it enables one to stand on
the shoulders of those who came before. I accomplished more than I had
expected in far, far less time than I had anticipated because I was able to
read and re-use the code written by my fellows and precursors. I am
thoroughly happy with my results and have been using this code to speed up
prototyping of Node.js code. The entire source code can be found here.
What instigated this post?
Last night, an unnamed redditor asked the WoW sub-reddit what the fastest way
to level these days is. Why? Because their girlfriend "has been wanting to
start playing wow with me". Seems reasonable, right? S/he goes on to ask about
RaF.
I immediately jump in and try to head off a disaster in the making. "What
disaster?" one may ask. Simple: RaF dungeon spamming isn't fun. In fact, I
wrote that "Personally, I wouldn't even use RaF because of how it completely
screws up the structure of the early game." This set the gears in my head to
whizzing frantically. What changed that made a really cool system actively
harm the game? And – more importantly – how can it be fixed?
What is Recruit-a-Friend?
In order to answer those questions, it is important to understand what the RaF
system actually does. Blizzard's FAQ does a good job of describing the
system. There are actually a lot of perks to using RaF, but there is one in
particular that really hurts the game: triple XP.
For levels 1 - 85, while in a group and similarly leveled the recruiter and
recruitee gain 3 times the normal amount of experience. This isn't simply mob
kill experience either: quest experience is also affected. The result these
days is that – if you aren't spamming dungeons to power-level – you
out-level zones just as you're starting to get their stories. To understand
the impact of this effect, we need to first dig deeper into what the reward
structure for WoW is.
I Saved Westfall and all I got was this stupid T-Shirt!
World of Warcraft is not unique in its structure. You help people, kill
monsters and collect rewards. There are two general classes of rewards in
WoW:
- Power-increasing rewards
These rewards increase the player's overall power level (although perhaps
not immediately). Examples of this are loot (literal character power), gold
(economic power) and experience (character power – albeit slightly
delayed).
- Emotional rewards
These rewards tug on the player's heart-strings. Whether it's saving an
adorable little orphan boy or laughing maniacally as you help Theldurin punch Deathwing in the face, these ones make you feel good (or bad) for
having done whatever it was you did. Type 1 rewards are a subset of this
reward class.
In my experience, the latter are much more important than the former. This is
upheld by observations of the reaction to the Madness of Deathwing fight and
Deathwing in general. While players got more powerful than ever before, there
was something missing. Emotional reward was lacking, and it showed.
How does this relate to Recruit-a-Friend?
The RaF system increases the gain rate of a particular Type 1 reward:
experience. However, it not only causes problems with the rate of gain of
other Type 1 rewards, but often outright prevents the gain of Type 2 rewards!
Recently, I leveled through Darkshore. Starting at level 11, I finished the
quest achievement at level 24. Had I been using RaF, I'd have only made it
through the first 1/3rd of the quests in that time. This would have left the
story hanging and broken the illusion of world-changing impact that Blizzard
has worked so hard to create.
As a result, emotional investment can become a liability preventing enjoyment
rather than a boon aiding it. It's like reading the first third of every
Spider-Man comic in order to 'catch up' to the current. Sure, one would reach
your goal faster, but at the cost of enjoying the process of reading comic
books. Even once you were caught up, you wouldn't understand all of the stuff
going on in the current issue.
I've seen situations where one player wants to get their significant other
into the game using RaF. In every case I've seen where the core benefit of RaF
is used to its fullest (ie by dungeon spamming), the SO quits playing.
Therefore, I believe that the overall benefit of RaF for the new player is
non-existent and in many cases it even causes damage to their perception and
enjoyment of the game.
Two Birds, One Stone
The solution to this problem is relatively simple. While simply removing the
XP bonus would go a long way towards preventing the damage currently being
done by RaF, why stop at simple prevention when it can be used to make the
game genuinely more enjoyable?
Think back, ye die-hard WoW fans: what problem always crops up when questing
as a group? Yes, that one. You know it well. Someone plays while the others
are away, gets ahead in both experience and quests and is either forced to
wait for the group to catch up, retread the content you just did, or leave the
group behind.
With long-time players, this isn't much of a problem. We have alts, we have
mains, and we can always do something else while the group is offline. For a
new player, however, such options are severely lacking. PvP grants experience,
dungeons grant experience, even gathering mats to level crafting grants
experience these days! Imagine if the Priest class is the only one that really
clicks with your friend. Are you going to ask them to not play when you aren't
online? To roll an alt? A second priest?
This problem is solved relatively well by the combination of massively
boosted XP and level granting: the increased XP rate encourages moving on to
other quest chains with relative frequency and level granting ensures that the
older player can keep up (most of the time). However, if triple XP is removed
from the system, then the problem again rears its ugly head because the player
no longer has such an incentive to move on in the middle of a quest chain.
Sure, the two players can remain evenly leveled, but what about quest
progress? Forcing the new player to retread content is not exactly ideal, so
why not allow the new player to catch the older one up not only in levels but
also in quests?
What I am proposing is this:
- Remove 3x XP from RaF
- Allow the new player to set the old player's quest progress to be equivalent
to their own
This would prevent XP gain from completely overriding any other sort of reward
in the game and would allow new players to continue questing with their
friends without worrying about quest dependencies and level discrepancies. To
my view, this would be superior to the current system – especially since the
store is now the go-to way to pay for a fast 90. However, one question remains
to be answered.
Why was it designed this way in the first place?
World of Warcraft is not the game that it once was. In ye olden days, when
Azeroth was yet young and paladins still only had 2 buttons for the first 40
levels, there were fewer quest chains and it was common – up til Outland, at
least – to complete a zone without having out-leveled it. In that era, there
were far fewer tales of merit told in the quests.
Way back then – near a full 6 years ago – tripling the experience rate made
sense. It meant that you'd have to do one zone to get through a level range
instead of 2.5-3. Still, those days are gone and now, with the world designed
to take one player through a level range in one zone, it no longer makes
sense.
Here's hoping that Blizzard fixes this system soon. It bothers me to think of
the people potentially missing a great experience because something that
should be rewarding can easily become the opposite. With all of the dramatic
WoD changes incoming, this could be the perfect time to do it!
For a long time I've had my blog hosted on Wordpress.com, but today that has
come to an end. While I've had a domain and server set up since mid-2013, I
hadn't had the opportunity to decide how to build my new blog. However, when
stasis was announced at the end of January, I realized that I may have found my
solution.
First off: why not Wordpress?
Nothing against Automatic, but after having run several WP blogs I sympathize
with this guy:
I don't want to run another WP blog and I don't want to have to hack any more
PHP. The solution?
Static Site Generation
Static Site Generation is a pretty simple concept. You have some templates and
some content, you want to put the content in the templates, and only want to
do so once. The site content is transformed into HTML once by the site owner
(aka me) and then served without any extra work by the server.
This has some big advantages. First, it makes a very fast website, as the
restriction is not HTML generation time but simple transmission time. Second,
it is extremely secure because malicious content serving is impossible short
of someone gaining root access on my server (or someone hijacking Disqus; I
trust Disqus' security people to do better than I could – it is their job,
after all).
Even better, because the code doesn't need to interact with the server, I am
not restricted to things that play nicely with the server (which Clojure
actually does through Java, but that's not a place I'd like to go right now).
I only had one big requirement – that I be able to write my posts in Org
format – but I also wanted something that I could hack on. Clojure is the
language I'm most interested in right now, so I started looking in that
direction. I toyed around with several options – even going so far as to fork
nakkaya's static – but eventually settled on magnar's stasis.
The biggest problem I had with static was how it dealt with posts. This
snippet says it best:
(defn list-files [d]
(let [d (File. (dir-path d))]
(if (.isDirectory d)
(sort
(FileUtils/listFiles d (into-array ["markdown"
"md"
"clj"
"cssgen"
"org"
"html"]) true)) [] )))
(defn create-latest-posts
"Create and write latest post pages."
[]
(let [posts-per-page (:posts-per-page (config))
posts (partition posts-per-page
posts-per-page
[]
(reverse (list-files :posts)))
pages (partition 2 (interleave (reverse posts) (range)))
[_ max-index] (last pages)]
...))
As you can see, the posts list is created by using partition on what
amounts to a directory listing. While this isn't a huge problem, my blog posts
aren't organized that way and I didn't want to change that. Having dates in
the file name looks ugly to me – never mind the fact that it duplicates the
#+DATE headers that are in all of my posts.
This is where stasis comes in. It's a no-batteries-included framework, which
means basically all it does is apply the templates to my sources. This leaves
designing the templates, template framework and sources to me. I used the
whattheemacsd source as my stasis-basis and built from there.
The biggest thing I had to do was implement conversion of Org files into
HTML. While not the fastest option (in terms of running time), I opted to
simply leave that to emacs by calling it in batch mode. The #+STUFF
headers are trivial to parse using regexp, so pulling in my #+DATE's was a
non-issue.
Ultimately, I'm pretty happy with how things turned out. This is the first
post I've written using the new system and it's worked great!
What next?
There are a couple of features that I want to build, starting with category
and tag views. After that, I may look at implementing an elisp command to
replace my current deployment method (a shell script) so that I can deploy
directly from the editor.
Technology & Style Credits
The full source code is available on github.