Breaking down the key elements of a good name is difficult. It seems to be more of an art than a science. The boundary between quite a good name and a very good name is fuzzy and liable to subjective opinions.
Consider a function that is responsible for applying multiple CSS styles to a button.
Imagine a scenario in which this is a standalone function. Which of the following names would you consider to be most suitable?
- styleButton
- setStyleOfButton
- setButtonCSS
- stylizeButton
- setButtonStyles
- applyButtonCSS
You’ve likely picked your favorite. And there is, amongst the readers of this book, bound to be disagreements. Many of these disagreements will be founded in our own biases. And many of our biases will have been conditioned by factors like what language we speak, what programming languages we’ve been previously exposed to, and what types of programs we spend our time creating. There are many variances that exist between all of us, and yet somehow, we have to come up with a non-fuzzy concept for what a “good” or “clean” name is. At the very least, we can say that a good name might have the following characteristics:
- Purpose: What something is for and how it behaves.
- Concept: Its core idea and how to think about it.
- Contract: Expectations about how it works.
This doesn’t completely cover the complexity of naming, but with these three characteristics, we have a starting point. In the remainder of this section, we will learn how each of these characteristics is vital to the process of naming things.
Purpose
A good name indicates purpose. Purpose is what something does, or what something is. In the case of a function, its purpose is its behavior. This is why functions are typically named in the verbal form, like getUser or createAccount, whereas things that store values are usually nouns, like account or button.
A name that encapsulates a clear purpose will never need further explanation. It should be self-evident. If a name requires a comment to explain its purpose then that is usually an indicator that it has not done its job as a name.
The purpose of something is highly contextual and so will, therefore, be informed by the surrounding code and the area of the codebase in which that name resides. This is why it’s often okay to use a generic name as long as it is surrounded by context that helps to inform its purpose. For example, compare these three method signatures within the class TenancyAgreement:
class TenancyAgreement { // Option #1: saveSignedDocument( id, timestamp ) {} // Option #2: saveSignedDocument( documentId, documentTimestamp ) {} // Option #3: saveSignedDocument( tenancyAgreementSignedDocumentID, tenancyAgreementSignedDocumentTimestamp ) {} } |
There are subjectivities to this, of course, but most people would agree that, when we have a surrounding context that communicates its purpose well, we shouldn’t need to granularize the naming of every variable within that context. With this in mind, we can say that Option #1 in the preceding code is too limited and may invite ambiguity, and option #3 is needlessly verbose, as parts of its argument names are already provided by its context. Option #2, however, with documentId and documentTimestamp, is “just right“: it sufficiently communicates the purpose of the arguments. And this is all we need.
Purpose is absolutely central to any name. Without a description or an indication of purpose, a name is merely decoration, and can often mean that users of our code are left rummaging around between documentation and other pieces of code just to figure something out. So we must remember to always consider whether our names communicate purpose well.
Concept
A good name indicates concept. A name’s concept refers to the idea behind it, the intent in its creation, and how we should think about it. For example, a function named relocateDeviceAccurately tells us not only what it will do (its purpose) but it informs us about the concept surrounding its behavior. From this name we learn that devices are things that can be located, and that the locating of such devices can be done at different levels of accuracy. A relatively simple name can arouse a rich concept within the minds of those who read it. This is part of the vital power of naming things: names are avenues to understanding.
A name’s concept, like its purpose, is strongly tied to the context in which it exists. Context is the shared space that our names exist within. The other names that surround the name we’re interested in are absolutely instrumental in helping us understand its concept. Imagine the following names seen together:
- rejectedDeal
- acceptedDeal
- pendingDeal
- stalledDeal
By these names, we immediately understand that a deal is something that can have at least four different states. It is implied that these states are mutually exclusive, and cannot apply to a deal at the same time, although that is unclear at this time. We are likely to assume that there are specific conditions related to whether a deal is pending or stalled, although we’re not sure what those conditions are. So, even though there is an ambiguity here, we are already starting to build up a rich understanding of the problem domain. That’s just by looking at names–without even reading the implementation.
We have spoken about context as a kind of shared space for names. In programming vernacular, we usually say that things named together in one area occupy a single namespace. A namespace can be thought of as a place where things share a conceptual area with each-other. Some languages have formalized the concept of a namespace into its own language construct (often called a package, or simply: a namespace). Even without such formal language constructs, JavaScript still makes it possible to construct namespaces via hierarchical constructs like objects. For example:
const app = {}; app.transactions = {}; app.transactions.dealMaking = {}; app.transactions.dealMaking.states = [ 'REJECTED_DEAL', 'ACCEPTED_DEAL', 'PENDING_DEAL', 'STALLED_DEAL' ]; |
Most programmers tend to think of namespaces as a very formal construct, but it’s not so. Often without knowing it, we are composing implied namespaces when we write functions with functions within them. Instead of being delineated by a level of an object hierarchy, the “namespaces” in this case are delineated by the scopes of our functions. For example:
function makeFilteredRequest(endpoint, filterFn) { return fetch(`/${endpoint}/`) .then(response => response.json()) .then(data => data.filter(filterFn); } |
Here we are making a request to an endpoint, via fetch, and before we return we are first gathering the required data via tapping into the Promise returned by fetch. To do this we use two then(…) handlers.
N.B. A Promise is a natively provided class that provides a useful abstraction for handling asynchronous actions. You can usually identify a promise by its then method, like that used within the preceding code. It’s common practice to either use promises or callbacks when tapping into asynchronous actions. You can read more about this in Chapter 10, “Control Flow” under the heading “Asynchronous Control Flow“.
Our first then(…) handler names its argument response, and the second one names its argument data. Outside the context of makeFilteredRequest, these terms would be very ambiguous. Yet because we are within the implied ‘namespace’ of a function related to making a filtered request, the terms response and data are sufficient to communicate their concepts.
The concepts communicated by our names, much like their purposes, are heavily intertwined with the contexts in which they are specified, so it’s important to consider not only the name itself but everything that surrounds it: the complex mesh of logic and behavior in which it resides. All code deals with some level of complexity, and conceptual understanding of that complexity is crucial in being able to harness it: so, when naming something, it helps to ask yourself: How do I want them to understand this complexity? This is relevant whether you’re crafting a simple interface to be consumed by other programmers, writing a deeply embedded hardware driver, or creating a GUI for non-programmers to consume.
Contract
A good name indicates a contract with other parts of the surrounding abstraction. A variable, by its name, may indicate how it will be used, or what type of value it contains, and what general expectations we should have about its behavior. It’s not usually thought about, but when we name something we are, in fact, setting up a series of implicit expectations or “contracts” that will define how people understand and use that thing. Here are some examples of the hidden contracts that exist in JavaScript:
- A variable prefixed with ‘is‘, for example, isUser, is expected to be a boolean type (either true or false).
- A variable in all-caps is expected to be a constant (only set once and immutable), e.g. DEFAULT_USER_EXPIRY
- Variables named plurally (e.g. elements) are expected to contain one or more items in a set-like object (e.g. an Array), whereas singularly named variables (e.g. element) are only expected to contain one item (not in a set).
- Functions with names beginning with get, find or select (etc.) are usually expected to return something to you. Functions beginning with process, build or run are more ambiguous and may not do so.
- Property or method names beginning with an underscore, such as _processConfig, are usually intended to be internal to an implementation or pseudo-private. They are not intended to be called publicly.
Whether we like it or not, all names carry with them the baggage of unavoidable expectations regarding their values and behaviors. It’s important to be aware of these conventions so that we do not accidentally break the contracts that other programmers rely on. Every convention will have an exception where it doesn’t apply, of course, but nonetheless, we should try to abide by them where possible.
Unfortunately, there isn’t a canonical list where we can find all of these contracts defined. They are usually quite subjective and will depend a lot on the codebase. Nonetheless, where we do encounter such conventions, we should seek to follow them. As covered in the second chapter, ensuring familiarity is a great way to increase the maintainability of our code. And there is no better way to ensure familiarity than adopt conventions that other programmers have come to adopt.
Many of these implied contracts are related to types, and JavaScript, as you may be aware,
is dynamically typed. This means the types of values will be determined at runtime, and
the type contained by any variable may be liable to change:
var something; something = 1; // a number something = true; // a boolean something = []; // an array something = {}; // an object |
The fact that a variable can refer to many different types means that the contracts and conventions implied by the names we adopt are even more important. There is no static- type checker to help us. We are left alone at the chaotic whim of ourselves and other programmers.
N.B. Later in the chapter, we discuss Hungarian Notation, a type of naming that is useful in dynamically-typed languages. Also, it’s useful to know that there are various static-type-checking and type-annotating tools available for JavaScript, if you find dealing with its dynamism painful. These are covered in Chapter 15 (Tools for Cleaner Code).
Contracts are not only important because of JavaScript’s dynamically-typed nature. They are fundamentally useful in giving us confidence in how certain values behave and what we can expect from them throughout the runtime of our program. Imagine if there was an API with a method called getCurrentValue() that didn’t always return the current value. That would break its implied contract. Seeing names through the lens of contracts is quite a mind-warper. Soon you will begin to see contracts everywhere — contracts between variables, between interfaces, and at the integration level between entire architectures and systems.
Now that we’ve discussed the three characteristics of a good name (Purpose, Concept, Contract) we can begin to explore some anti-patterns, i.e. ways of naming things that we should try to avoid.
This was an excerpt from my new book, “Clean Code in JavaScript“.
]]>Severe events can focus the mind. I want to make sure I’m doing things I (a) want to do, and (b) believe will have a net-positive impact on my local community and the wider world.
I’ve decided to try at an application for YearHere, a program for social impact in which students explore social problem domains and develop novel solutions through businesses. Examples include: CrackedIt: a business that employs ex-offenders and at-risk youth, reducing crime & post-offense economic risks and recidivism. And BirdSong: a fashion brand selling clothes & accessories sourced ethically and made by women who are paid a living wage.
My recent experience with a physical disability and my historical struggles with ADHD and mental health have helped to focus my heart and mind on the issue of economic participation amongst those who are ill, disabled, or otherwise unable to commit to the demands of the 40-hour work week and its associated struggles. What follows is the short essay I submitted as part of my application to YearHere.
Both paid and volunteering roles are often advertised with an expectation of a fixed hourly commitment. However, many individuals aren’t able to participate due to health reasons or other difficulties. This group includes a cross-section of society: disabled individuals, caregivers, the recently retired, the mentally ill, the chronically ill, and the neuro-divergent. Many of these people are left sidelined by the demanding 40+ hour work week and daily challenges including mobility, caregiving responsibilities, learning difficulties, cognitive differences, fatigue, and pain.
Even with its flaws, the current working week was a victory of sorts, won over time by activism and unionization during the industrial revolution. Amongst this drive for progress, however, there have always been cynical economic interests. Henry Ford supported less working hours because it meant more “consuming” hours, in service of economic growth. There have been efforts to further reduce the working week around the world but often the aim is to maximise economic indicators, but not social indicators such as “inclusion” or “equality”.
There is widening disability employment and pay gap in the UK[1][6]. Isolation and lack of economic participation creates observable negative effects on health [2][5]. Disabled people more often end up in food poverty, are less educated, are more likely to experience hate crime or harassment, and have more issues with housing and transport [4][6]. By lifting up these groups via economic and community participation, we can slowly improve these inequalities.
As a disabled person myself, and as someone with a history of ADHD and mental-ill-health, I have experienced some of the struggles of keeping up with typical expectations of work. I have witnessed how people are actively disabled by society. We prevent equal access to opportunities just because certain groups may be less able to fulfil our expectations of time, energy, or “normal behaviour”. In principle, these challenges are accounted for in the Equality Act 2010. It implores employers to reduce “substantial disadvantages” experienced by disabled individuals [7], but what do we do when the very fabric of our working society produces such disadvantages? Let’s also not forget that there are vast swathes of individuals who may not be considered disabled but still have significantly challenging differences or deficits. Even in the general population, the vast majority of working adults would prefer more flexible working [1].
I believe we can create frameworks for flexible working and tools that allow sidelined groups to partake “at will” in the community and economy without having to commit to fixed schedules and expectations. Further political and legislative change to bolster disability rights is, of course, needed, but I believe we can augment these efforts with novel approaches in how we channel participation in our local communities and economy.
Fundamentally, we need to find ways to project the presence, skills and perspectives of sidelined groups into society and the local economy. As Dr Lisa Cameron MP stated [3], “work confers important benefits. It provides opportunity for purposeful activity, for financial independence, for social inclusion and social status.”
References: https://gist.github.com/padolsey/702cb6c54ad102654e476e525ac15046
]]>
This is not a book about frameworks or “one size fits all” abstractions. It’s about navigating, with care, the needs of our users and fellow programmers. The world of JavaScript is increasingly complex, burdened and blessed with myriad opinions and approaches. I wrote this book, in part, because I wanted to distill this complexity.
The book is over 500 pages and covers topics across the gamut from DOM reconciliation to the Law of Demeter. The broad chapters are outlined here:
- What is Clean Code Anyway?
- Setting the Scene
- The Tenets of Clean Code
- The Enemies of Clean Code
- SOLID and Other Principles
- Naming Things Is Hard
- JavaScript and Its Bits
- Primitive and Built-In Types
- Dynamic Typing
- Operators
- Parts of Syntax and Scope
- Control Flow
- Crafting Abstractions
- Design Patterns
- Real-World Challenges
- Testing and Tooling
- The Landscape of Testing
- Writing Clean Tests
- Tools for Cleaner Code
- Collaboration and Making Changes
- Documenting Your Code
- Other Peoples’ Code
- Communication and Advocacy
- Case Study (Exploration of a real-world project)
The book functions as both a user-empathic exploration of programming and a nitty-gritty JavaScript (ES2020) manual, training the reader to consider how each design decision and language construct might help or hinder the quality of their code and the experiences it creates.
Take a look: Clean Code in JavaScript (Publisher Link)
]]>I’ve tried to recount below the distinct skills that I feel I lacked most sorely when attempting to take my hobbyist mindset into the working world. The following reflections are, of course, heavily biased to my own experiences. I’ve worked within projects with only a couple of people, startups from 8 to 200+ people, and corporates as large as 50,000, and what I’ve written reflects that journey. Your mileage may vary.
People and communication
Communicating, enquiring, pushing back, asking crucial questions, and sharing insights. Doing all of these without too much ego, and without over-biasing to your own opinions, is very tricky. In the past I often mistook passion for some kind of license to impose opinion. Passion is a wonderful energy, but it can give rise to recklessly dismiss others’ very valid thoughts. Humility, kindness and listening are hard to master, but worth dedicating time towards.
Updates and deadlines
Being asked “how long will this take?” always terrified me. I never had a good answer. I’m better now, in that I take the approach of “Take however long you initially think, then double it”. Even with this, you’ll usually be wrong. The more extraneous or chaotic factors are involved, the more fuzzy the estimate will have to be. Keeping people updated is key. Truly understanding the needs of PMs, managers and all other stakeholders will make your life 1000x easier (and theirs!).
Code comprehension
This, for me, was and remains to this day the greatest challenge. Truly understanding the code written by others — internalizing it as if it were my own — is incredibly hard. I never figured out how to be good at this. Learning through test suites or reverse engineering via a UI or CLI were my favorite approaches until I had to deal with spaghetti architectures with hundreds of classes. There is no magical elixir. The answer is probably time, focus, and some special ‘something’ that I haven’t yet found. Crafting the right questions to ask people is likely crucial here. But sometimes, at least for me, my confusion is so severe that I cannot articulate where to even begin.
Just making it work
For perfectionists, this is hell. But it’s important. Perfect is the enemy of good. The increasingly marginal benefits yielded on the journey towards perfection are rarely worth it anyway. You may not get to work on the things you most want to, but if you are strategic you can carve out a role where you’re able to. And in this role you can give your perfectionism an outlet.
Sharing knowledge and helping others
This comes naturally to some. Other people find this difficult. I tend to love sharing knowledge, but only if I feel confident in my understanding of it. If I lack confidence, I’d rather shut-up and say nothing. but this is also a trap, because in many places of work it’s rare for anyone to know all the things anyway. It’s better to share what you know humbly, hopeful that someone else knows more, than to share nothing at all. Seeking help and helping others isn’t always about Q&A; it’s about spotting gaps in confidence or knowledge and working with them to bolster those areas. I feel like pair-programming (or even group-programming) is an often sidelined but incredibly valuable tool for this.
Performing to others’ metrics
A lot of the time you will value your work according to your own biased reasoning and passions. Sometimes these won’t align with how you’re being judged for that work though. You’ll be pushed and tugged towards towards understanding your work through metrics and expectations that you may not agree with. Finding a place in your heart and mind where you can reconcile these differences is important. Passion, again, can distract you from the sometimes less exciting truth: that you are employed to produce output aligned with an economic motive. Crucially, if you can find a place of work whose metrics and values truly align with yours, your life and work will be happier and more fulfilling.
Making it work for you
The hardest lesson. And one that I often reflect on. There are many types of career available in the world of making software. Finding one that works for you is hard. Don’t lose hope, and don’t be afraid to explore hybrid roles or even other types of work entirely. Life is short. The world is large.
]]>Through helping to teach these courses I’ve observed the successes and difficulties that students are having, and I’d like to share some of these observations below. These are mostly limited to the teaching of the web platform — that is: HTML, CSS and JavaScript, which are the core pieces that we cover when someone has no prior coding experience.
Teaching HTML, CSS and JavaScript, in that order, makes a tonne of sense. Ordering a curriculum according to layers of abstraction is a very obvious idea, but it’s worth stating it outright as such. It’s easy to forget the necessity of foundational knowledge, and the usefulness of not only understanding how a new thing works, but where it is situated in the universe of related paradigms. Knowing CSS and HTML isn’t the same as knowing how to piece them together in useful ways.
Libraries and frameworks are double edged swords when learning. Bootstrap and jQuery have been really useful to us in getting quick prototypes working, and making the web platform more palatable to beginners, but the band-aid has to be ripped off at some point, because after a while the abstractions start to break down. Learning via first principles, instead, would perhaps make people more resistant to the pain of these leaky abstractions and the ideologies and illusions created by them.
Relatedly, it’s impossible to teach the right thing. In this ever-changing world of web things it’s very hard to know what paradigm(s) to teach. For example, the once sacred virtue of ‘Separation of Concerns’ for HTML/CSS/JS is becoming less relevant as we find more interesting and optimal ways to create DOM and style it (See React). The blasphemy of inline styles, and inline UI logic, is fast becoming defunct. The ebbs and flows of our trends and paradigms are hard to predict. All we can do is teach what is current, with a bias towards foundational knowledge. (I.e. don’t start off by teaching React or Angular…)
Syntax is not intuitively delineated. As you pick up new programming languages, mentally parsing different types of syntax becomes second nature (an awesome feat of the human mind). Your brain can build a hierarchical model of the instructions you’re seeing. The CSS value is part of the CSS rule which is part of the CSS ruleset (etc). But I’ve found that these models aren’t always intuitive. It’s not obvious from the get-go that an HTML element should be closed with a </blah> instead of a <blah/> or a /<blah>. And just because it’s easy to look at a piece of syntax and understand its meaning doesn’t mean its easy to intuitively come up with the correct syntax when you know what you want to do (“I want to close this HTML element — how do I do that again? Something related to a forward-slash I think…”). Just think about all the different types of symbols we use… is it always absolutely obvious which of them function as delimiters, versus operators, versus arbitrary values? It’s absolutely ~!:@${/?[{]};:~ not.
Ordering of instructions is not an obvious concern. The idea that you have to write your HTML or CSS or JavaScript in a specific order is not obvious to someone who hasn’t coded before. It has to be specifically explained. Markup and other declarative languages especially often conflate complex concepts with ordering (it’s not just a case of procedural execution). In CSS we worry about the cascade and specificity. In HTML we worry about semantics and how elements will be visually ordered. There’s a lot of complexity in deciding what goes before or after something else.
The web platform makes it easy to win AND fail. You can build amazing things very quickly with these basic tools, but HTML and CSS don’t tell you when you fail. They silently try to do their best, and are very flexible when it comes to some syntactic variations, but strict when it comes to others. Usually no errors will be thrown in the console either. So when your page isn’t displaying correctly you have to pour over every piece of the code to discover what might be causing the problem. A simple “CSS: Unexpected colon on line 17” would be a huge help to those learning. This to me seems like a reason to work harder on educational platforms for the web, with linters and other aids built in, perhaps. Right now the state of the Web Inspector as a learning aid is not too wonderful. It’s a powerful tool, but also a necessarily complex tool tailored to professionals.
]]>I, for one, like a central mutuality and the joy of freeform conversations. Quizzes and prescriptive interactions aren’t for me. But then… everyone’s different.
Offer them a drink. This shouldn’t need explanation. It’s just part of being a good host. Let them know where the facilities are too. Generally be polite. Think about that feeling of being utterly welcomed and comfortable in a new setting, with new people. Try your very best to emulate that.
Let them decide where to sit. This might seem a pointless gesture but it helps loosen the implicit power dynamic that exists. Making them comfortable eases the pressure and communicates your intent to have an informal and exploratory conversation. This is how you will learn most about them, and vice-versa.
Don’t take notes or recite from their CV. A CV in-hand, or a laptop, or an open notebook directly in front of you communicates an interrogatory dynamic. Making a person feel judged is rarely the best way to get to know them. This counts for both the interviewer and interviewee. Of course, take as many notes as you like after the interview.
Get to know them – have a sincere conversation about them, you, the company, the world. Don’t be rigid or prescriptive. Be open to the natural ebbs, flows and tangents of conversation.
Ask them what’s most important to them in a job. Ask them what the perfect job would look like. Really listen. Pay attention. Ask them what the world lacks, and what it has too much of. Share your own opinions passionately and positively.
Describe a problem you’re having — at work — and work through some possible solutions, however they wish to (talk, whiteboard, code, operatic verse, whatever). Make this a collaborative effort. It has to be a real unsolved problem. This will exemplify their working technique, and yours, and is probably the closest you’re going to get to knowing what it’s like to work with each-other.
Don’t quiz them. It’s patronising and sets in stone a dynamic of process and judgment. It’s usually better to move fluidly through a mutually intriguing conversation than to have a set definition of what you want to talk about. Prescriptive conversations are incredibly limited. There’s rarely a lot of signal gained in the mindless recollection of rote learning. Also, demanding a specific discussion will make them feel as if they have to delicately wind their way through a very specific and unknown maze of statements to win your approving nods and hmms and yesses. It’s a fruitless game for both parties.
Welcome tough questions. Be honest. Don’t pretend that everything is rainbows and daisies. Tell them of your personal experience at work — the ups and downs… the real stuff. If they’re hired they’ll eventually know the good and the bad anyway. The best people will recognise the value in your honesty and will reciprocate in kind.
]]>It is precarious up here, on this plateau of abstraction, with no easy way down and only the option to build up, and beyond, to an even more unsteady structure on which others will have to balance.
Others: the users, sitting atop this stack with but a cursor to direct its magical ways.
And I am the same. I am the user, only my cursor is more peculiar and adept, and through it I may express logic and behaviour to a finer degree.
But I am still the user inasmuch as the user is the programmer of some higher purpose.
The distance between us is only the foggy cliffs of abstraction and the misdirection of software. What distinguishes us is our altitude; on what steep mountain we wearily sit.
]]>My only gripe: Google imposes an email alias limit of 30. I’m a person who tends to use a new alias for every new subscription, store, account or occasion, so 30 aliases simply doesn’t cut it. More than 30 seems silly to many people; they opt to just use their primary alias and a + separated suffix, like '[email protected]'. This works fine, and depending on the client you can also use other separators, but there are three major hindrances as far as I see it:
- Many apps don’t allow emails containing
+before@, usually because they've blindly adopted the first (incorrect) email validation regex they came across. - Many people don’t understand emails that go too much adrift from
[a-zA-Z0-9_-]in the alias portion. I’ve often had people literally writing down my email asjamesplusfoo@...instead ofjames+foo@..., which is understandable, but means I have to always say “james, << the actual PLUS character >>, foo, at, …”. - Apps that understand the
+suffix trick will also understand that it can, with many email clients, be eliminated or replaced. This means nasty spammers could automate the generation of additional+aliases to avoid you blacklisting any single address.
Number 3 isn’t much of a concern, really, but I reckon of all address aliasing tricks, the + suffix technique is one of the most transparent and therefore one of the most open to abuse by spammers.
Gmail in Google Apps does provide a “catch-all” option, so {anything}@yourdomain.com would get received. The problem with this is that it leaves you totally exposed to random-alias-spewing spammers. They can scatter-gun your email very successfully. I’ve experienced this at least once before, when I wasn’t on Google, but even without precedent I’d be worried enough. Google makes it quite clear that they’ll readily close accounts that go over their receiving limits.
For aliases, it’s probably better to have a well constructed whitelist than a blacklist that requires constant maintenance. In this spirit, I was pleasantly surprised to find that Gmail does provide a way to modify “default routes” if you’re the domain admin. So, really, the 30 alias limit never applied to me, I just didn’t know it. The routing configuration also supports regular expressions, which is fantastic!
(Go to Apps > Google Apps > Settings for Gmail > Advanced settings > Default routing)
This means, in addition to my regular aliases (up to 30 of them), I can configure it such that specifically formatted recipient addresses flow into my inbox quite naturally, while addresses that don’t fall within my stringent format will get rejected.
What format? It doesn’t matter. As long as its obscure enough to avoid automated scatter-gunning, I’m happy. If it’s also obscure enough to avoid real-human malice then I’d be over the moon.
Unfortunately, Google only allows a limited subset of the regular expressions we’re used to (they use a regex implementation called RE2). No backtracking or backreferencing of any kind is allowed. This makes it near-impossible to construct overly clever formats… And this is a shame, because my dream in life is an email alias format so mathematically confusing that even sober-me cannot solve it unless I'm sufficiently invested in the provision of an email address. This would, for example, prevent me from politely giving out my address, willy nilly, to store clerks, websites, and random people on the street.
Alas, given this limited subset of regex I’ve had to construct a much more lenient format. This is the kind of thing I'm talking about:
-.+\d\d@: Must include at least one dash and end in two digits.
E.g. [email protected].^[a-zA-Z].*\d@: Must start in a letter and end in a digit.
E.g. [email protected].^(store|account|legal|banking)-.+@: Must start in one of the listed prefixes.
E.g. [email protected].
After deciding on the format you can choose to 'modify message' and simply set the envelope-recipient to your primary email address.
If you wanted to be super paranoid you could set-up additional MX records for a wildcard subdomain, and then just have a fixed alias of your name with infinite subdomains:
You could also construct yourself a little bookmarklet or addon that spits out hashes or numbers of a specific type that can be checked in your regular expression. And for when you get the rare joy of sharing an email address in real life you could take inspiration from 2FA and have a card of a dozen ready-to-go addresses. I think this may be a step too far though…
]]>Tanzania, [U][n][i]ted Republic of[U][n][i]ted Arab Emirates[U][n][i]ted Kingdom[U][n][i]ted StatesT[u][n][i]sia
Of course, it’s probably not the last one, but that right there is a human intuition that we often forget to instil into these UI interactions.
We can divine what the user probably intends most of the time but it'll always be a game of heuristics. Most solutions shy away from this game, opting instead to match the query letter-for-letter in each potential value, and this is usually sufficient, but without any other logic not only will “la” match “Latvia” but also “Angola”. And usually “Ltvia” will match nothing whatsoever, even though it’s seemingly obvious what the user is trying to type.
If you try implementing a fuzzy matcher to solve this, the first revelation is that you can't just boolean-match the query against the data like so many solutions do. You need to score each potential match. Hopefully, in the case of country selection, you end up with a sensible subset of countries that match the query to some reasonable degree. This scoring is necessary so that you know what you're putting at the top of the list. When typing "U", the user expects Ukraine or Uzbekistan sooner than Mauritius or Sudan, for example.
Oddly, if you looked at the most common autocompletion widget out there (jQuery UI), it doesn't appear to follow this intuition.
Even the most graceful solutions tend to avoid the muddiness of dealing with mistakes like “untied states” or “leichtenstein”. Sure, the likeliness of a person having to type the name of a country they aren’t intimately familiar with is probably quite low, but people still make mistakes.
I've been intrigued by this topic for quite a while and it's why I originally made relevancy.js. It solves the problem quite well, I think, and it does so in a pretty transparent way, with scores applied for various qualities such as the index of the query within the target string ("king" scores higher than "dom" in "kingdom", for example), but it's still a quite a lot of code for such a tiny part of an overall user experience.
I have once again been playing with this problem (thanks to a certain tweet) and have so wanted to come up with something stupefyingly graceful.
It all starts with a scratch in back of your mind — the one that tells you that your time has come. The world requires you to use regular expressions.
Warning: I don’t sincerely recommend doing any of this. It’s just a bit of fun. It’s probably an inefficient, unreliable, obscure and ultimately foolish endeavour!
Let’s begin!
A static France might look like this:
/^France$/ |
A more lenient France might be less sensitive to its case:
/^france$/i |
We could then allow the characters to be optional too:
/^f?r?a?n?c?e?$/i |
This would match “f” and “franc” and “FaE”, etc.
But… users make even more grievous mistakes sometimes, and our regular expression should be able to handle those. So let’s add a single character of leniency between each legitimate character, and at the beginning and end of the string:
/^.?f?.?r?.?a?.?n?.?c?.?e?.?$/i |
But then this would allow contiguous mistakes like “fafafafa”. We only want to allow a *single* incorrect mistake after each successfully entered character. For this we can use groups to force each character to be matched and a lazy quantifier on the mistake character to ensure that legitimate characters get to successfully match.
So:
/f.?otherStuff/ |
Becomes:
/(?:f.??)?otherStuff/ |
In English: Try to match f followed by otherStuff. If impossible then try to match any character after f but before otherStuff. (This is why lazy quantifiers (e.g. ??) are so useful!)
The entire regex would become:
/^(?:.(?=f))?(?:f.??)?(?:r.??)?(?:a.??)?(?:n.??)?(?:c.??)?(?:e.??)?$/i |
We should probably capture each individual match (f should be (f)) so that we can analyze the result and score it appropriately.
var r = /^(?:(f).??)?(?:(r).??)?(?:(a).??)?(?:(n).??)?(?:(c).??)?(?:(e).??)?$/i 'f$R-aN_cEx'.match(r); // => ["f$R-aN_cEx", "f", "R", "a", "N", "c", "E"] |
The regular expression, broken down:
/
^ # Start of string
(?: # Non-captured group
(f) # Match and capture 'f'
.?? # Followed lazily by any character
)? # Entire group is optional
(?: # Non-captured group
(r) # Match and capture 'f'
.?? # Followed lazily by any character
)? # Entire group is optional
... # Etc.
$ # End of string
/i |
A quick note: lazy or lazily in the context of regular expressions simply means that that thing will be intentionally excluded from the first match attempt and will only be used if the subsequent regular expression is unsuccessful without it.
One caveat with the above regex is that it doesn’t allow a mistake to be at the beginning of the string. We could fix this with a lookahead to the effect of “allow a mistake here as long as its followed by a non-mistake” but since “non-mistake” could effectively be any character in the legitimate string it’s easier to just make allowances for that initial mistake in each group. Additionally, we probably want to capture every single mistake, in addition to legitimate characters. Here’s our next iteration:
/
^ # Start of string
(?: # Non-captured group
(^.)? # Captured optional mistake at the beginning of the string
# ===============================================
(f) # Match and capture 'f'
(.??) # Followed lazily by any character (captured)
)? # Entire group is optional
... # Etc.
$ # End of string
/i |
The check (^.)? has to be specified in each group, to account for mistakes that don’t involve “f”, like “krance” or “ttance”, etc.
Since we’re aiming to genericize this entire mess, we should create a generator that assembles the regular expression given any piece of text:
function makeFuzzyRegex(string) { if (!string) { return /^$/; } // Escape any potential special characters: var cleansed = string.replace(/\W/g, '\\$&'); return RegExp( '^' + cleansed.replace( // Find every escaped and non-escaped char: /(\\?.)/g, // Replace with fuzzy character matcher: '(?:(^.)?($1)(.??))?' ) + '$', 'i' ); } makeFuzzyRegex('omg'); // => /^(?:(^.)?(o)(.??))?(?:(^.)?(m)(.??))?(?:(^.)?(g)(.??))?$/i |
This regex matched against ‘_o-m*g!’ produces:
[ // Full match: "_o-m*g!", // Captures: "_", // Mistake "o", // Legit "-", // Mistake undefined, // Void mistake "m", // Legit "*", // Mistake undefined, // Void mistake "g", // Legit "!" // Mistake ] |
The captures are in groups of three, with every second capture being the legitimate character (case-insensitive), and with every first and third potentially being mistakes.
We can then loop through these captures and apply weights as we see fit.
var fullMatch = makeFuzzyRegex('omg').exec('_o-m*g!'); var captures = fullMatch.slice(1); // Get captures specifically var score = 0; for (var i = 0, l = captures.length; i < l; i += 3) { if (captures[i]) score -= 1; if (captures[i+1]) score += 10; if (captures[i+2]) score -= 1; } score; // => 26 |
That scoring is quite arbitrary, but we’ve at least prescribed our wish to score successes more than we punish mistakes (10 vs 1).
We can start to play with the heuristics of this if we wrap it all up:
function createFuzzyScorer(text) { var matcher = makeFuzzyRegex(text); return function(query) { var match = matcher.exec(query); if (!match) return 0; var captures = match.slice(1); var score = 0; for (var i = 0, l = captures.length; i < l; i += 3) { if (captures[i]) score -= 1; if (captures[i+1]) score += 10; if (captures[i+2]) score -= 1; } return score; }; function makeFuzzyRegex(string) { if (!string) { return /^$/; } // Escape any potential special characters: var cleansed = string.replace(/\W/g, '\\$&'); return RegExp( '^' + cleansed.replace( // Find every escaped and non-escaped char: /(\\?.)/g, // Replace with fuzzy character matcher: '(?:(^.)?($1)(.??))?' ) + '$', 'i' ); } } |
Our first attempt isn’t too bad:
var score = createFuzzyScorer('omg'); score('omg'); // => 30 score('xOmg'); // => 29 score('.o.m.g.'); // => 26 score('om'); // => 20 score('og'); // => 20 score('o'); // => 10 score('nope'); // => 0 |
These seem like sensible enough scores, generally, but we’re more interested in autocompletion, and so there’s an obvious predictive element there. If a user types ‘o’ then that should probably score higher than ‘g’ if we’re testing against ‘omg’, but with the above mechanism they both receive a standard 10:
var score = createFuzzyScorer('omg'); score('o'); // => 10 score('g'); // => 10 |
We can fix this by applying a higher weight to matches that appear earlier in the string:
// The scoring loop: for (var i = 0, l = captures.length; i < l; i += 3) { if (captures[i]) score -= 0.1; if (captures[i+1]) score += (l - i) / l; // the magic if (captures[i+2]) score -= 0.1; } |
Now the score given for any singular legitimate match will decrease as the index (i) increases. Here are the results:
var score = createFuzzyScorer('omg'); score('omg'); // => 1.99 score('xOmg'); // => 1.90 score('om'); // => 1.66 score('.o.m.g.'); // => 1.59 score('og'); // => 1.33 score('o'); // => 1.00 score('nope'); // => 0.00 |
This is getting closer to our intuition. The next step would be to try to create a real autocompletion widget. I’ve done it so I know that we’ll want to make one more change. The problem with our scoring right now is that it’ll award legitimate characters relative to the length of the string. But when comparing scores across multiple subject strings, this approach seems broken.
createFuzzyScorer('RuneScript')('Ru'); // 1.9 createFuzzyScorer('Ruby')('Ru'); // 1.7 |
These should both score equally, as “Ru” is just as likely to become “Ruby” as it is to become “RuneScript”. To achieve this we should only take into account the index, and make the weight of any scoring decision inversely proportional to that index, in this case via an exponential taper (pow(index, -2)).
// The scoring loop: for (var i = 0, l = captures.length; i < l; i += 3) { var relevancyOfCharacter = Math.pow(i + 1, -2); if (captures[i]) score -= relevancyOfCharacter * 0.1; if (captures[i+1]) score += relevancyOfCharacter * 1; if (captures[i+2]) score -= relevancyOfCharacter * 0.1; } |
(Final version of createFuzzyScorer available as a gist.)
See this demo using programming languages as the dataset. Try intentionally misspelling something (jawascript), or missing out characters (jaascit), or just going a little crazy (jahskt). It works beautifully.
To achieve speedy sorting, a fuzzy scorer is created for every single value before the user types anything:
var data = PROGRAMMING_LANGUAGES.map(function(lang, i) { return { actualValue: lang, score: createFuzzyScorer(lang), i: i, toString: function() { return lang; } }; }); |
This means we can iterate through data on every relevant input event, and call the score() method with the current query. We can then bundle this into a filter->sort->slice flow to get our list of sensible suggestions:
var sorted = data.filter(function(item) { // Get rid of any very unlikely matches (and cache the score!) return (item._cachedScore = item.score(query)) >= .5; }).sort(function(a, b) { var as = a._cachedScore; var bs = b._cachedScore; // Sort by score, and if score is equal, then by original index: // (We would typically return 0 in that case but some engines don't stable-sort) return as > bs ? -1 : as == bs && a.i < b.i ? -1 : 1; }).slice(0, 10); // We only need the top 10... |
And.. we’re done. It’s never really finished though: you’ll find endless tweaks that can be made to the scorer to make it more believably resemble human-like intuition.
For those wanting to test the resulting country autocompletion interaction: See the demo.
I guess, despite my initial warning, I wouldn’t actually mind using this in production, as long as there were a decent number of unit tests. I’d probably also assemble the regular expressions on the server and serve them up as literals. It’s also worth mentioning that almost everything in this post has been exploring the fuzzy-matching of very short strings in small datasets. Even in the case of the country demo, to get more applicable results, I broke up long names into the component parts and then scored against each. E.g.
// E.g. Macedonia, the Former Yugoslav Republic of: var scorers = [ "Macedonia, the Former Yugoslav Republic of", "Macedonia", "the", "former", "yugoslav", "republic", "of" ].map(createFuzzyScorer); // Etc. |
And this would be terribly inefficient on a larger scale, so with any dataset longer than a list of countries you’re probably best to explore Trie-based approaches to autocompletion.
And with that, I’ll shut-up and wish you merry regex’ing!
]]>The sea of things I don’t know, even within an area so confined as ‘UI development’ or ‘JS' is now too much to reasonably handle. So, like everyone, I pick a select group of things to focus on, and I pray that those chosen don't inflate beyond attainability or die of obsolescence before I can grasp them.
The ever mutating ‘front end’ field is now a massive treasure chest of things I don’t know. There’s just too much candy to handle. It’s very well, knowing Backbone or Knockout, but before long you need Angular too, and Ember… and React… and Polymer. You tell yourself, it’s cool, you have a solid grasp of the guiding principles behind these abstractions so you needn’t bother with their nuanced APIs… but actually, wait, this isn’t your grandmother’s MVC or MVVM or MVP, and your productivity depends on those nuances. Almost everything is opinionated, so you must agree or in abstinence risk obsolescence.
You’ll invest a couple months learning that thing you should have learned a year ago only to be told, “oh didn’t you hear? Those pre-compilers are defunct… and inline styling is now a best practice!”, or some equally disappointing variation.
"The world is changing. Keep learning!"
This is fair, but learning is hard, especially if you’re only chasing the curve so as to keep up with it. Surely, you should want to keep up because the knowledge itself is enriching? And maybe that’s the crux? How can you divine useful knowledge from passing fads or needlessly niche lexicons? It doesn’t so much feel like knowledge attainment. It feels like an API. And another API. And another one. Dozens of competing abstractions and concepts, opinionated and divided. Pick one, and pray it’s the right one. Or hedge your bets by half-knowing several.
The early days: Grit
What originally sold me on all this lovely stuff was the grit, the scrappiness, the openness to hacking. It was a very low barrier to entry, with a real sense of healthy cluelessness and worthy toil. I loved it. There were very few best practices. Very little was solved, and for every problem there were a hundred solutions that held their water… Nowadays, every library or framework has its own logo, a conference, an army of employed open-source contributors pushing its agenda. For a technical niche, this is what maturity feels like, maybe. There is still a persisting love of grit and hacking, but it feels sidelined or masked beneath layers of marketable-framework’y-kool-aid.
I miss hacking around, and when buy-in was cheap. Despite this I still love the wonder and passion of it all though, and while the modus operandi of this niche is now one of professionalism above amateurism and certainty above scrappiness, I will always try to engage with the grit.
]]>