Birthday retrospective ¶

- November 30, 2016

So, recently Evennia celebrated its ten-year anniversary. That is, it was on Nov 20, 2006, Greg Taylor made the first repo commit to what would eventually become the Evennia of today. Greg has said that Evennia started out as a "weird experiment" of building a MUD/MUX using Django. The strange name he got from a cheesy NPC in the Guild Wars MMORPG and Greg's first post to the mailing list also echoes the experimental intention of the codebase. The merger with Twisted came pretty early too, replacing the early asyncore hack he used and immediately seeing a massive speedup. Evennia got attention from the MUD community - clearly a Python-based MUD system sounded attractive.

When I first joined the project I had been looking at doing something MUD-like in Python for a good while. I had looked over the various existing Python code bases at the time and found them all to be either abandoned or very limited. I had a few week's stunt working with pymoo before asking myself why I was going through the trouble of parsing a custom script language ... in Python ... Why not use Python throughout? This is when I came upon Evennia. I started making contributions and around 2010 I took over the development as real life commitments forced Greg to step down.

Over the years we have gone through a series of changes. We have gone from using SVN to Mercurial and then to using GIT. We have transited from GoogleCode to GitHub - the main problem of which was converting the wiki documentation (Evennia has extensive documentation).

For a long time we used Python's reload() function to add code to the running game. It worked ... sometimes, depending on what you changed. Eventually it turned out to be so unpredictable that we now use two processes, one to connect clients to and the other running the game, meaning we can completely restart one process without disconnecting anyone.

Back in the day you were also expected to create your own game in a folder game/ inside the Evennia repo itself. It made it really hard for us to update that folder without creating merge conflicts all over. Now Evennia is a proper library and the code you write is properly separated from ours.

So in summary, many things have happened over the years, much of it documented in this blog. With 3500 commits, 28 000 lines of code (+46% comments) and some 25 people contributing in the last year, Openhub lists us as

"A mature, well-established codebase with a stable commit history, a large development team and very well documented source code".

It's just words compiled by an algorithm, but they still feel kinda good!

While Evennia was always meant to be used for any type of multiplayer text game, this general use have been expanded and cleaned up a lot over the years.

This has been reflected in the width of people wanting to use it for different genres: Over time the MUSH people influenced us into adding the option to play the same character from many different clients at the same time (apparently, playing on the bus and then continuing on another device later is common for such games). Others have wanted to use Evennia for interactive fiction, for hack&slash, deep roleplay, strategy, education or just for learning Python.

Since Evennia is a framework/library and tries to not impose any particular game systems, it means there is much work to do when building a game using Evennia. The result is that there are dozens of games "in production" using Evennia (and more we probably don't know about), but few public releases yet.

The first active "game" may have been an Evennia game/chat supporting the Russian version of 4chan... The community driven Evennia demo-game Ainneve is also progressing, recently adding combat for testing. This is aimed at offering an example of more game-specific code people can build from (the way Diku does). There are similar projects meant for helping people create RPI (RolePlay Intensive) and MUSH-style games. That said, the Evennia-game Arx, After the Reckoning is progressing through beta at a good clip and is showing all signs of becoming the first full-fledged released Evennia game.

So cheers, Evennia for turning 10. That's enough of the introspection and history. I'll get back to more technical aspects in the next post.

Season of fixes ¶

- October 13, 2016

The last few months has been dominated by bug-fixing and testing in Evennia-land. A lot more new users appears to be starting to use Evennia, especially from the MUSH world where the Evennia-based Arx, After the Reckoning is, while still in alpha, currently leading the charge.

With a new influx of users comes the application of all sorts of use- and edge-cases that stretch and exercise the framework in all the places where it matters. There is no better test of code than new users trying to use it unsupervised! Evennia is holding up well overall but there are always things that can be improved.

• I reworked the way on-object Attributes was cached (from a stupid but simple way to a lot more sophisticated but harder way) and achieved three times faster performance in certain special cases people had complained about. Other issues also came to view while diving into this, which could be fixed.
• I reworked the venerable batch command and batchcode processors (these allow to create a game world from a script file) and made their inputs make more sense to people. This was one of the older parts of Evennia and apart from the module needing a big refactoring to be easier to read, some parts were pretty fragile and prone to break. Especially when passing it file names tended to be confusing since it understood only certain relative paths to the files to read in and not even I could remember if one should include the file ending or not. This was cleaned up a lot.
• Lots of changes and updates were made to the RPSystem contrib, which optionally adds more advanced roleplaying mechanics to Evennia. The use of this in Evennia's demo game (Ainneve, being separately developed) helps a lot for ironing out any remaining wrinkles.
• Lots and lots of other fixes and smaller feature updates were done (About 150 commits and 50 Issues closed since end of summer).

A fun thing with a growing user base is that we are also starting to see a lot more Pull requests and contributions. Thanks a lot, keep 'em coming!

• Map system contrib (merged), for creating a world based on ASCII map. Talking about maps, users contributed not just one but two new tutorials for implementing both static and dynamic maps with Evennia.
• Webclient notifications (pending), for making the webclient show us in a clearer way when it gets updated in a different tab. A more advanced implementation awaits the webclient being expanded with a proper client-side option window; there is currently a feature request for this if anyone's interested in taking it on.
• AI system contrib (pending). This is a full AI backend for adding complex behaviors to game agents. It uses Behavioral trees and is designed to be modified both in code and from inside the game.
• Action system contrib (pending). This contrib assigns the actions of Characters a time cost and delays the results of commands the given time. It also allows players to go into turn-based mode to enforce a strict action order.
• Lots of now closed PRs were contributed by the Arx lead developer to fix various bugs and edge-cases as they came up in live use.

The fixing and tightening of the nuts and bolts will likely continue the remainder of the year. I'm currently working on a refactoring of the way command sets are merged together (see the end of my blog post on Evennia in pictures for a brief summary of the command system). But with so much new blood in the community, who can tell where things will turn from here!

The art of sharing nicks and descriptions ¶

- July 1, 2016

In the month or so since the merger of Evennia's development branch and all its web-client updates, we have been in bug-fixing mode as more people use and stress the code.

There have been some new features as well though - I thought it could be interesting to those of you not slavishly following the mailing list.

Shared web login

When you are logged into the website you will now also auto-login to your account in the web client - no need to re-enter the login information! The inverse is also true. You still need to connect to game at least once to create the account, but after that you will stay connected while the browser session lasts.

Behind the scenes the shared login uses cookies linked to server-side Django sessions which is a robust and safe way to manage access tokens. Obviously browser sessions are irrelevant to telnet- or ssh connections.

Extended Nicks

Evennia's nick(name) system is a way to create a personal alias for things in game - both to on-the-fly replacing text you input and for referring to in-game objects. In the old implementation this replacement was simply matched from the beginning of the input - if the string matched, it was replaced with the nick.

In this new implementation, the matching part can be much more elaborate. For example you can catch arguments and put those arguments into the replacement nick in another order.

Let's say we often use the @dig command this limited way:

@dig roomname;alias = exit;alias, backexit;alias

Let's say we find this syntax unintuitive. The new nick system allows to change this by catching the arguments in your nick and put it into the "real" command. Here is an example of a syntax putting the aliases in parentheses and separating all components with commas:

> nick newroom $1($2), $3($4), $5($6) = @dig $1;$2 = $3;$4, $5;$6

From here on you can now create your rooms with entries like this:

> newroom The great castle(castle), to the castle(castle), back to the path(back)

Multidescer contrib

I have added a new "multidescer" to the contrib folder. A multidescer is (I think) a MUSH term for a mechanism managing more than one description. You can then combine any of these various descriptions into your "active" description.

An example of usage:

desc hat = a blue hat.
desc basic = This is a tall man with narrow features.
desc clothing = He is wearing black, flowing robes.

These commands store the description on the Character and references them as unique keywords. Next we can combine these strings together in any order to build the actual current description:

> desc/set basic + |/ + clothing + On his head he has + hat
> look self
This is a tall man with narrow features.
He is wearing black, flowing robes. On his head he has a blue hat.

This allows for both very flexible and easy-to-update descriptions but also a way to handle freeform equipment and clothing. And you can of course use the nick system to pre-format the output

> nick setdesc $1 $2 $3 $4 = $1 + |/ + clothing + On his head he has a $4

This way you can clothe yourself in different outfits easily using the same output format:

> setdesc basic clothing hat

The multidescer is a single, self-contained command that is easy to import and add to your game as needed.

... There's also plenty of bug fixes, documentation work and other minor things or course.

Anyway, summer is now upon us here in the northern hemisphere so things will calm down for a bit, at least from my end. Have a good 'un!

Evennia in Pictures ¶

- May 31, 2016

This article describes the MU* development system Evennia using pictures!

This article was originally written for Optional Realities.

Since it is no longer available to read on OR, I'm reposting it in full here.

Figure 1: The parts of the Evennia library

Evennia is a game development library. What you see in Figure 1 is the part you download from us. This will not run on its own, we will soon initialize the missing “jigsaw puzzle” piece on the left. But first let’s look at what we’ve got.

Looking at Figure 1 you will notice that Evennia internally has two components, the Portal and the Server. These will run as separate processes.

The Portal tracks all connections to the outside world and understands Telnet protocols, websockets, SSH and so on. It knows nothing about the database or the game state. Data sent between the Portal and the Server is protocol-agnostic, meaning the Server sends/receives the same data regardless of how the user is connected. Hiding behind the Portal also means that the Server can be completely rebooted without anyone getting disconnected.

The Server is the main “mud driver” and handles everything related to the game world and its database. It's asynchronous and uses Twisted. In the same process of the Server is also the Evennia web server component that serves the game’s website. That the Server and webserver are accessing the database in the same process allows for a consistent game state without any concerns for caching or race condition issues.

Now, let’s get a game going. We’ll call it mygame. Original, isn’t it?

Figure 2: The full setup for mygame

After installing evennia you will have the evennia command available. Using this you create a game directory - the darker grey piece in Figure 2 that was missing previously. This is where you will create your dream game!

During initialization, Evennia will create Python module templates in mygame/ and link up all configurations to make mygame a fully functioning, if empty, game, ready to start extending. Two more commands will create your database and then start the server. From this point on, mygame is up and running and you can connect to your new game with telnet on localhost:4000 or by pointing your browser to http://localhost:4001.

Now, our new mygame world needs Characters, locations, items and more! These we commonly refer to as game entities. Let’s see how Evennia handles those.

Figure 3: The Database Abstraction of Evennia entities

Evennia is fully persistent and abstracts its database in Python using Django. The database tables are few and generic, each represented by a single Python class. As seen in Figure 3, the example ObjectDB Python class represents one database table. The properties on the class are the columns (fields) of the table. Each row is an instance of the class (one entity in the game).

Among the example columns shown is the key (name) of the ObjectDB entity as well as a Foreign key-relationship for its current “location”. From the above we can see that Trigger is in the Dungeon, carrying his trusty crossbow Old Betsy!

The db_typeclass_path is an important field. This is a python-style path and tells Evennia which subclass of ObjectDB is actually representing this entity.

Figure 4: Inheriting classes to customize entities

In Figure 4 we see the (somewhat simplified) Python class inheritance tree that you as an Evennia developer will see, along with the three instanced entities.

ObjectDB represents stuff you will actually see in-game and its child classes implement all the handlers, helper code and the hook methods that Evennia makes use of. In your mygame/ folder you just import these and overload the things you want to modify. In this way, the Crossbow is modified to do the stuff only crossbows can do and CastleRoom adds whatever it is that is special about rooms in the castle.

When creating a new entity in-game, a new row will automatically be created in the database table and then “Trigger” will appear in-game! If we, in code, search the database for Trigger, we will get an instance of the Character class back - a Python object we can work with normally.

Looking at this you may think that you will be making a lot of classes for every different object in the game. Your exact layout is up to you but Evennia also offers other ways to customize each individual object, as exemplified by Figure 5.

Figure 5: Adding persistent Attributes to a game entity.

The Attribute is another class directly tied to the database behind the scenes. Each Attribute basically has a key, a value and a ForeignKey relation to another ObjectDB. An Attribute serializes Python constructs into the database, meaning you can store basically any valid Python, like the dictionary of skills seen in Figure 5. The “strength” and “skills” Attributes will henceforth be reachable directly from the Trigger object. This (and a few other resources) allow you to create individualized entities while only needing to create classes for those that really behave fundamentally different.

Figure 6: Sessions, Players and Objects

Trigger is most likely played by a human. This human connects to the game via one or more Sessions, one for each client they connect with. Their account on mygame is represented by a PlayerDB entity. The PlayerDB holds the password and other account info but has no existence in the game world. Through the PlayerDB entity, Sessions can control (“puppet”) one or more ObjectDB entities in-game.

In Figure 6, a user is connected to the game with three Sessions simultaneously. They are logged in to their Player account Richard. Through these Sessions they are simultaneously puppeting the in-game entities Trigger and Sir Hiss. Evennia can be configured to allow or disallow a range of different gaming styles like this.

Now, for users to be able to control their game entities and actually play the game, they need to be able to send Commands.

Figure 7: Commands are Python classes too

Commands represent anything a user can input actively to the game, such as the look command, get, quit, emote and so on.

Each Command handles both argument parsing and execution. Since each Command is described with a normal Python class, it means that you can implement parsing once and then just have the rest of your commands inherit the effect. In Figure 7, the DIKUCommand parent class implements parsing of all the syntax common for all DIKU-style commands so CmdLook and others won’t have to.

Figure 8: Commands are grouped together in sets and always associated with game entities.

Commands in Evennia are always joined together in Command Sets. These are containers that can hold many Command instances. A given Command class can contribute Command instances to any number of Command Sets. These sets are always associated with game entities. In Figure 8, Trigger has received a Command Set with a bunch of useful commands that he (and by extension his controlling Player) can now use.

Figure 9: Command Sets can affect those around them

Trigger’s Command Set is only available to himself. In Figure 8 we put a Command Set with three commands on the Dungeon room. The room itself has no use for commands but we configure this set to affect those inside it instead. Note that we let these be different versions of these commands (hence the different color)! We’ll explain why below.

Figure 10: The name Command “Set” is not just a name

Command Sets can be dynamically (and temporarily) merged together in a similar fashion as Set Theory, except the merge priority can be customized. In Figure 10 we see a Union-type merger where the Commands from Dungeon of the same name temporarily override the commands from Trigger. While in the Dungeon, Trigger will be using this version of those commands. When Trigger leaves, his own Command Set will be restored unharmed.

Why would we want to do this? Consider for example that the dungeon is in darkness. We can then let the Dungeon’s version of the look command only show the contents of the room if Trigger is carrying a light source. You might also not be able to easily get things in the room without light - you might even be fumbling randomly in your inventory!

Any number of Command Sets can be merged on the fly. This allows you to implement multiple overlapping states (like combat in a darkened room while intoxicated) without needing huge if statements for every possible combination. The merger is non-destructive, so you can remove cmdsets to get back previous states as needed.

… And that’s how many illustrations I have the stamina to draw at this time. Hopefully this quick illustrated dive into Evennia helps to clarify some of the basic features of the system!

Evennia 0.6! ¶

- May 22, 2016

As of today, I merged the development branch to make version 0.6 of the MU* development system and server Evennia.

Evennia 0.6 comes with a lot of updates, mainly in the way Evennia talks to the outside world. All communication is now standardized, so there are no particular treatment of things like text - text is just one of any standardized commands being passed between the server the client (whether over telnet, ssh, websockets or ajax/comet).

For example the user can now easily plug in "inputfuncs" to handle any data coming from the client. If you want your client to offer some particular functionality, you just need to plop in a python function to handle it, server-side. We also now offer a lot of utility functions for things like monitoring change (tell the client whenever your health status changes so it can update a health bar or flash the screen).

The HTML5 webclient has itself updated considerably. Most is happening behind the scenes though. Notably the webclient's javascript component is split into two:

• evennia.js, acts as a library for handling all communication with the server part of Evennia. It offers events for a gui library to plug into and send/receive. It will also gracefully degrade from a websocket connection to AJAX/COMET long-polling if the player uses an older browser.
• evennia_gui.js is the default front-end and implements a traditional and stable "telnet-like" interface. The html part uses uses Django templating to make it easy to customize and expand. Since this simply makes use of the resources offered by evennia.js, one could pretty easily slip in some other gui library here, or set up calls to get all sorts of interesting information from the server (which talks back using inputfuncs).

There are a truckload of more updates and features that are covered on the mailing list.

Technical stuff happening ¶

- March 24, 2016

Hi folks, a bit more technical entry this time. These usually go onto the Evennia mailing list but I thought it would be interesting to put it in the dev-blog for once.

So, I'm now halfway through the TODO list issue of the wclient development branch as alluded to in the last post. The wclient branch aims to rework and beef up the web client infrastructure of Evennia.

The first steps, which has been done a while was converting the SSH/SSL and IRC input/output protocols to use the new protocol infrastructure (telnet and websockets was done since before). That's just under-the-hood stuff though. Today I finished the changes to the Monitor/TickerHandlers, which may be of more general interest.

With the changes to the the way OOB (Out-Of-Band) messages are passing through Evennia (see this mailing list post for more details), the OOBHandler is no more. As discussed there, the handling of incoming data is becoming a lot freer and will be easily expandable to everyone wanting to make for a custom client experience. The idea is thus for Evennia to offer resources for various input commands to make use of, rather than prescribing such functionality in a monolothic way in the OOBHandler. There were three main functionalities the OOBHandler offered, and which will now be offered by separate components:

1. Direct function callback. The instruction from the client should be able to trigger a named server-side function. This is the core of the inputfunc system described previously.
2. Field/Attribute monitoring. The client should be able to request monitoring of an object's database fields or Attributes. For example, the client may request to be notified whenever the Character's "health" Attribute changes in some way. This is now handled by the new monitorhandler. See below.
3. Non-persistent function repeats. One should be able to set up a repeating ticker that survives a server reload but does not survive a cold shutdown - this mimics the life cycle of server Sessions. Scripts could do this already but I wanted to be able to use the TickerHandler for quick assignment. Problem was that the Tickerhandler in master branch is not only always-persistent, it also only calls database object methods. So I have now expanded the tickerhandler to also accept arbitrary module functions, without any connection to a database object.

The MonitorHandler

evennia.MONITOR_HANDLER is the new singleton managing monitoring of on-object field/attribute changes. It is used like this:

 MONITOR_HANDLER.add(obj, field_or_attrname, callback, **kwargs)

Here obj is a database entity, like a Character or another Object. The field_or_attrname is a string giving the name of a db_* database field (like "db_key", "db_location" etc). Any name not starting with db_ is assumed to be the name of an on-object Attribute (like "health"). Henceforth, whenever this field or attribute changes in any way (that is, whenever it is re-saved to the database), the callback will be called with the optional kwargs, as well as a way to easily get to the changed value. As all handlers you can also list and remove monitors using the standard MONITOR_HANDLER.remove(), .all() etc.

The TickerHandler

evennia.TICKER_HANDLER should be familiar to Evennia users from before - it's been around for a good while. It allows for creating arbitrary "tickers" that is being "subscribed" to - one ticker will call all subscribers rather than each object or function having its own timer.

Before, the syntax for adding a new ticker required you specify a typeclassed entity and the name of the method on it to call every N seconds. This will now change. This is the new callsign for creating a new ticker:

 TICKER_HANDLER.add(interval, callback, idstring="", persistent=True, *args, **kwargs)

Here**, interval,** like before, defines how often to call **callback(*args, kwargs).

The big change here is that callback should be given as a valid, already imported callable, which can be either an on-entity method (like obj.func) or a global function in any module (like world.test.func) - the TickerHandler will analyze it and internally store it properly.

idstring works as before, to separate tickers with the same intervals. Finally persistent=False means the ticker will behave the same way a Script with persistent=False does: it will survive a server reload but will not survive a server shutdown. This latter functionality is particularly useful for client-side commands since the client Session will also not survive a shutdown.

... So this is a rather big API change to the TickerHandler, which will mean some conflicts for those of you relying heavily on tickers. Easiest will definitely be to simply stop the old and start new ones. It's not clear yet if we'll offer some automated way to convert old tickers to new ones. Chime in if this is something important to you.

Happening Next

The next steps involves making use of these new utilities to implement the basic OOB commands recommended by the MSDP and GMCP protocols along with some recommended functionality. We'll see how long that takes, but progress is being made. And if you are a web guy, do consider helping out.

Climbing up Branches ¶

- February 14, 2016

Today I pushed the latest Evennia development branch "wclient". This has a bunch of updates to how Evennia's webclient infrastructure works, by making all exchanged data be treated equal (instead of treating text separately from other types of client instructions).

It also reworks the javascript client into a library that should be a lot easier to expand on and customize. The actual client GUI is still pretty rudimentary though, so I hope a user with more web development experience can take upon themselves to look it over for best practices.

A much more detailed description of what is currently going on (including how to check out the latest for yourself) is found in this mailing list post. Enjoy!