Useful Unicode resources

Over the years I’ve collected some good Unicode resources I might as well share.

Navigators and Directories

  • Facebook symbols, which organizes and lists symbols in categories for easy access to post on social media.
  • Fileformat.info, which has a bit of an old fashioned design but includes a ton of useful information on the specifics of each characters.
  • Unicode Lookup provides a handy search function to find particular characters.
  • Unicode Search, another search tool.
  • Unicode Table, which is similar to Unicodinator (see below), but provides a cool feature of showing a map of where on Earth different characters are used.
  • Unicode Table For You gives you some useful sliders to move around Unicode tables.
  • Unicodinator, a visual unicode navigator I created, allowing you to infinitely scroll through the entire multilingual plane (0 – DFFF). Very helpful if you are looking for a character and don’t know what precisely you are looking for.
  • Unify, a growing list of characters along with information about what devices and browsers can display them.

Articles and Education

JavaScript libraries

  • CharFunk is a library I wrote that provides some of the functionality that Java’s Character class does.
  • JS Codepoints extends String to allow you to count codepoints properly.
  • XRegExp provides RegExp functionality that works correctly with Unicode.

Other tools and resources

  • Alan Wood’s Unicode resources collects a bunch of useful links and information.
  • Shapecatcher is a super useful tool that lets you draw a symbol and then shows you the characters that match.
  • Unicode.org is the site of the Unicode Consortium and so is the final say for all things Unicode.
  • Unifoundry provides a GNU Unicode font to allow display of every code point in the Unicode BMP.

Let me know if I missed any other useful resources!

Singularity University at Taser International

On Monday I had the good fortune to attend a one day session of Singularity University (hereafter SU) hosted by Taser International. The speakers gave an amazing tour of the exponential growth occurring in most fields of technology, covering topics like artificial intelligence, big data, nanotechnology and business organization in this new era. Although I have been following in these topics for a long time, I learned an incredible amount from this conference. Especially because this Taser session was partially focussed on the law enforcement ramifications of all this, something I haven’t thought as much about. It was a fantastic experience that will leave me thinking for a long time, and one I would highly recommend.

The conference was kicked off by Rick Smith, CEO, director and co-founder of Taser, showing a clip of JFK talking about the moon shot plan.

Smith used the video to demonstrate that exponential progress is not a radical new concept. Smith himself has built a phenomenal, innovative company (with a men-in-black level futuristic office) built around leveraging technology in law enforcement. His commitment to positive innovation is demonstrated by the fact he hosted this conference in order to introduce his employees and customers to the idea of accelerating technological growth. I’m certain this will pay dividends for years to come.

The first speaker was Salim Ismail, entrepreneur and Global Ambassador of SU. He started by talking about the accelerating pace of change in digital technology, pointing out that if the speed of our cars had grown the same way computing speed had, cars would travel much faster than the speed of light by now. From there he proceeded to give a whirlwind sampling of accelerating change in biotech, autonomous vehicles, drones, and many other areas.

His examination of 3D printing was especially illuminating. We have reached a double inflection point in this technology. First, we can now print things that can’t be manufactured any other way. Second, unlike other manufacturing processes, with 3D printing there is no additional cost for more complex objects. This technology is only a few short years from disrupting industries as diverse as auto parts and pastry chefs (yes, we can print food now).

His conclusion was that when an industry becomes digitally based, exponential growth follows. Moore’s law is only one example. The biggest impediments to this change are governments (designed for a steady rate of change), and traditionally organized businesses (designed around creating predictability). This is not surprising since our brains are built to expect, at most, linear change, which was fine for most of our history. Part of SU’s mission is to change that. Their summer graduate program guides teams in envisioning and planning projects that will positively impact a billion people in ten years, such as Matternet and Modern Meadow.

Our next speaker was Neil Jacobstein, co-chair of the Artificial Intelligence and Robotics Track at SU, who spoke about breakthroughs in AI. Many things we used to believe only humans could do are now routinely handled by computers. The real promise of AI is not only that it will eventually be able to do everything that humans can do, but that it willl do things it would be impossible for humans to do. His dizzying tour of these applications included:

  • IBM’s Jeopardy-winning AI, Watson, which has now “graduated from play to work”, tackling real world problems. (I met one of the creators, Torsten Bittner, last year)
  • Pervasive involvement of AI in stock trading
  • Warehouse robots and intelligent distribution technologies
  • A simulation of a macaque monkey brain

His conclusion was that we will eventually be able to reverse engineer the brain and build human level AI. This will be not be used to replace us, but instead will be used to augment our brains (which have not seen a significant hardware upgrade in at least 50 thousand years). This process has already begun with smart phones giving us access to vast information and capabilities.

Next we heard from Ekho Inc‘s Kent Langley, who spoke about big data. He began by talking about the emerging “internet of things”, which will by 2020 be a $1.9 billion dollar market including 212 trillion connected devices. All these devices will be generating data that will need to be stored, analyzed and leveraged — primarily via AI. But today really big data is still really difficult to deal with.

In the face of this current difficulty, Langley has some concrete recommendations for businesses grappling with big data today. He also talked about all the open source tools available for big data related tasks today, including GNIP, Hadoop, Spark and others.

Marc Goodman of Future Crime spoke next, portraying the world of crime in an era of exponentially stronger and cheaper technology. According to Goodman, we are seeing a paradigm shift in crime. Sony experienced a massive network hack that cost them $171 million dollars. Mexican drug lords built their own private cellular network. Drones are used to fly cellphones and drugs into heavily guarded prisons. The list went on and on.

Despite all this, Goodman’s outlook on the future is hopeful. He closed with a long list of ways the good guys are using technology to fight crime, for example:

  • Robots being used to keep schools safe in Korea
  • Predictive policing technology used to focus law enforcement resources where they are needed before they are needed
  • Crowd sourced intelligence on riot crimes

Then Ismail gave a second presentation, this one about how companies and other groups can cope with a world of exponential growth and change. As the economy becomes more information based (and therefore continues to speed up) all kinds of business assumptions are upended at a rapid pace. Companies are either disrupters or disrupted. Organizations that flatten their hierarchy and distribute decision making have a better chance of survival and success. Companies that democratize the design, manufacturing and distribution of goods are best positioned for the future. Some examples Ismail used included:

Rick Smith spoke briefly about code.org (who’s co-founder Hadi Partovi is on Taser’s board) and the hour of code. He then introduced our keynote speaker, Ray Kurzweil.

Ray Kurzweil is an inventor, author, futurist and entrepreneur. His talk was titled “How to Create a Mind” (also a book) and it was every bit as interesting as I had hoped. He began by a deep exploration of the accelerating growth of computing power. Since 1962 we have seen a many billion fold increase in price/performance ratio of computing power. We’ve seen computing hardware get cheaper, smaller, faster and more powerful, all at a pace that accelerates every year.

He then examined how our brain evolved and how it works today. Science has made great progress recently in understanding the brain. For example we now know that we fill up our brain’s knowledge storing capacity by the time we are twenty, at which point we have to forget something in order to learn something new. Wouldn’t it be great if we could get around that?

Kurzweil believes we will eventually be able to do that and more, by extending our mind into the cloud. He was recently hired by Google to work on natural language comprehension, one of the remaining steps needed to build a digital mind. There is growing consensus in the AI world that we should reach human levels of artificial intelligence by about 2029 — a prediction Kurzweil made many years ago. From there, this AI can improve its own design at an exponentially growing rate. The results of that development are by nature unpredictable — which leads us to the concept of the Singularity.

After the conference was over, I was very honored to be able to attend a dinner with Kurzweil, as well as a few bright folks from Taser. It was definitely one of the most interesting dinner conversations I’ve ever taken part in. Kurzweil deserves his reputation as one of the great thinkers of our time.

There was a child in attendance for much of the day. Having two young, smart daughters of my own, I was intrigued to witness her interest and understanding of the topics covered. She asked a few questions during the Q&A periods which demonstrated her grasp of the material: “If you engineered a horse with the strength of a lion and speed of a cheetah, could you race it?” and “How far away are we from plugging Google into our heads?” Her generation is inheriting a world where asking these kinds of questions will be a daily necessity. I am glad they are up to the task.

Deep Dive of Mobile App and Mobile Service – Seattle Tech Forum

I’ve missed a few of these but I was able to get out to July’s STF, which was about mobile technologies. Our sponsor was PDS Tech, a recruiting and staffing company.

The first speaker was chief Strategy Officer at Ratio Interative, Russ Whitman. His talk was titled “Mobile is eating our world”. It was a tour of all the statistics showing how smartphone and tablet growth is outpacing PCs. He also talked about how mobile apps are changing our world. Since Ratio Interactive helps companies plan, build and deploy mobile apps it’s no surprise Whitman had a lot of good information.

Next we heard from Jason Clark, Facebook‘s Lead Software Engineer for the Mobile Platform team. He started out by having us all raise our hands with our thumbs up, which he took a photo of to prove we liked his talk. For his presentation, titled “Making Mobile Apps Social”, he walked us through adding Facebook actions to a simple Rock-Paper-Scissors iPhone app. It was pretty cool because I was able to look up one of his shares and like it on my iPhone — and see the little notification number on his demo screen almost immediately bump up by one.

Jason Clark's demo

Finally we heard a talk about “Mobile Computing in the Cloud” from Min Zhu, Principal Development Manager for Windows Phone at Microsoft. He gave a lot of good information about the Windows Phone store, how that service is put together based on Azure, and their efforts to grow it.

I’m now a Web Developer at Amazon

This past Monday I started my new position as a web developer at Amazon! So far I’ve mostly been training, but it looks like I will be able to do some actual productive work this coming week, which is pretty exciting. The culture shock of going from a very small company to an enormous one is definitely real, but not as bad as I expected. In fact, it reminds me a lot of college — working directly with many of the same people every day, but always meeting new smart people as well.

(Needless to say, nothing I have said or will say on this website is anything other than my own opinion, not Amazon’s…)

Anyway, I am really feeling great about this move!

JavaScript character utility CharFunk 1.1.0 released

CharFunk is a little library I wrote a few years ago to make it easier to do things with Unicode text. I revisited it recently to clean up and improve the code, and add tests and a few features. The API is pretty simple:

  • CharFunk.getDirectionality(ch) – Used to find the directionality of the character
  • CharFunk.getMatches(string,callback) – Returns an array of contiguous matching strings for which the callback returns true, similar to String.match()
  • CharFunk.isAllLettersOrDigits(string) – Returns true if the string argument is composed of all letters and digits
  • CharFunk.isDigit(ch) – Returns true if provided a length 1 string that is a digit
  • CharFunk.isLetter(ch) – Returns true if provided a length 1 string that is a letter
  • CharFunk.isLetterNumber(ch) – Returns true if provided a length 1 string that is in the Unicode “Nl” category
  • CharFunk.isLetterOrDigit(ch) – Returns true if provided a length 1 string that is a letter or a digit
  • CharFunk.isLowerCase(ch) – Returns true if provided a length 1 string that is lowercase
  • CharFunk.isMirrored(ch) – Returns true if provided a length 1 string that is a mirrored character
  • CharFunk.isUpperCase(ch) – Returns true if provided a length 1 string that is uppercase
  • CharFunk.isValidFirstForName(ch) – Returns true if provided a length 1 string that is a valid leading character for a JavaScript identifier
  • CharFunk.isValidMidForName(ch) – Returns true if provided a length 1 string that is a valid non-leading character for a ECMAScript identifier
  • CharFunk.isValidName(string,checkReserved) – Returns true if the string is a valid ECMAScript identifier
  • CharFunk.isWhitespace(ch) – Returns true if provided a length 1 string that is a whitespace character
  • CharFunk.indexOf(ch) – Returns the first index where the character causes a true return from the callback, or -1 if no match
  • CharFunk.lastIndexOf(ch) – Returns the last index where the character causes a true return from the callback, or -1 if no match
  • CharFunk.matchesAll(string,callback) – Returns true if all characters in the provided string result in a true return from the callback
  • CharFunk.replaceMatches(string,callback,ch) – Returns a new string with all matched characters replaced, similar to String.replace()
  • CharFunk.splitOnMatches(string,callback) – Splits the string on all matches, similar to String.split()

This allows you to do some things you would have a hard time doing in JavaScript otherwise. JavaScript RegExps are notoriously useless for dealing with non-ASCII data. For example, imagine you wanted to do something simple like replace all non-word characters with an underscore. This is easy:

"The United States of America".replace(/[^\w]/g,"_");
    //returns "The_United_States_of_America"

Unless of course, you are dealing with non-ASCII letters:

"Российская Федерация".replace(/[^\w]/g,"_"); 
    //returns "___________________" 
"جمهورية مصر العربية".replace(/[^\w]/g,"_"); 
   //returns "____________________"

That’s not what we want.

Fortunately, CharFunk can handle this using replaceMatches:

function notLetterOrDigit(ch) {
    return !CharFunk.isLetterOrDigit(ch);
}

CharFunk.replaceMatches("جمهورية مصر العربية",notLetterOrDigit,"_"); 
    // returns "جمهورية_مصر_العربية"

CharFunk.replaceMatches("Российская Федерация",notLetterOrDigit,"_"); 
   //returns "Российская_Федерация"

This is just one small example of what CharFunk can do. I hope that web developers working on international projects — which is pretty much any web app these days — will find this useful!

Cases of User Experience – Seattle Tech Forum

This month’s STF was all about User Experience. It was sponsored by boutique software house Webtellect. Webtellect creates custom software solutions, primarily based on Microsoft technologies.

Our first speaker was Christopher Johnson, a Designer at Google on the Hangouts team in Mountain View. His talk was titled “What the Hell is UX, Anyway?”. He explained how the technology market has shifted towards design-led companies. The obvious example is Apple, but you can also see this at Microsoft, Facebook, Path, Pinterest, Instagram, Google… the list goes on. The iPhone really kicked off this trend in 2007. Mobile has forced software creators to focus on simplicity and design — with a smaller screen, you have put more care into what you put on it.

This shift took hold at Google when Larry Page became CEO and rallied the company to create “one beautiful, intuitive user interface”. This effort became known as “Project Kennedy” and the results are evident across all of Google’s properties. This new emphasis on design brought Google’s users not only prettier, more consistent screens, but also a better User Experience.

According to Johnson, UX is a story. For a software user there is a problem to solve, and a beginning, a middle and an end. He then gave us five tips to becomming a UX focussed organization so that we can give the story of our user’s experience a happy ending:

  • Let your designers out of their cage
  • Prototype early and often
  • Treat design as an equal partner with engineering
  • Talk with and watch users
  • Embrace the design sprint

Johnson left us by answering the question in the title of his talk: What the hell is UX? UX is everyone’s responsibility.

Jeremy Foster, Developer Evangelist at Microsoft, gave a talk titled “Ultimate User Experience”. Primarily it was an illustration of design principles using Microsoft’s new metro UI.

Foster compared good design to the movement of air. Poor design is like air at rest — particles of air moving around in every direction are like a user with no clear indication of what to do. Good design, like a communicative sound pushing waves of air towards your ear, gives the user direction. He made a strong argument for Microsoft’s new edge-to-edge design language, listing principles like “content before chrome”, “minimize distractions”.

Every time I hear a passionate Microsofty explain the ideas behind the new UI and show it off, I feel like they really got it right. Windows 8 is beautiful and perfectly crafted to the consumer computing experience. I’m still not convinced it will fully succeed in a work environment, but I was still impressed by Foster’s talk.

Finally we heard from Charlie Claxton, VP of Creative Strategy at Produxs, with a presentation titled “The Habit of Design”. Claxton also gave an STF presentation on UX last July. He covered concepts such as framing, social proof, loss aversion and operant conditioning.

Maybe the most interesting part of his presentation centered around Eugene Pauly, an amnesiac who suffered from short term memory loss. Pauly could not remember what he did five minutes earlier or remember the layout of his new home. However, he was still able to learn in a way, by being made to repeat the same experience daily and thereby forming habits. Claxton claims that good design should be aimed at creating and leveraging habits.

This well-attended STF was very educational. Join us on April 17th for our next topic, “Legal Matters for Technology Company”.

If Business had Glass

Not many people are talking about how Google Glass could be used by business and industry. Google seems to be aiming Glass straight at consumers, with a design that is surprisingly sleek and marketing videos full of personal moments. Apple has proven that there are plenty of people who will eagerly spend money on the latest sexy gadget. Still, I think the enterprise might be an even better early adopter than hipsters with money to burn.

There are endless possibilities for Glass to create immediate business value. Almost any employee using an mobile device or tablet could be made more effective if their hands and eyes were left free to do real work.

  • A warehouse manager could use Glass to look at shelves and obtain immediate information about what is in stock, how fast product is moving, and more.
  • A forklift driver in that warehouse could use Glass to navigate the warehouse while keeping their hands on the controls.
  • A doctor could (with your permission and the proper ten pages of paperwork) record their view of an operation for future reference (and for liability protection).
  • A delivery driver could use Glass to safely navigate their route and deal with changes while they are driving.
  • A host on a cruise ship could access useful and timely information about their guests and provide quick answers to guest’s questions, all without breaking eye contact.
  • A repair person could use schematics and instructions as they work on fixing complex machinery, without ever setting down their tools.

These are just a few possibilities in an endless list, which will grow as the devices capabilities improve. I can imagine even more applications in government, athletics and entertainment.

Beyond the incredible possible applications, there are other reasons business and industry would be a great fit as an early adopter for Glass. Businesses get to skip over the issue of “will people want to wear these sci-fi looking things in public”. Businesses might have fewer qualms with the high early price point provided it creates compelling, measurable value — companies routinely spend upwards of $2k on ruggedized mobile devices (although cheaper Android and iOS solutions are competing here). Businesses can fund the development of useful applications without the need for bootstrapping or venture capital.

I’m still really excited about the consumer possibilities of Glass, including Games. But I hope that Google and Business get together and leverage Glass to bring new possibilities to enterprise computing.

Cases of Digital Marketing – Seattle Tech Forum

This week’s STF was sponsored by tech-talent firm Chameleon Technologies. The topic was Digital Marketing and it was extremely well attended, by a fairly different crowd than we normally see — lots of marketing people of course. This is a topic I am not very familiar with, so it was pretty interesting stuff.

Our first speaker was Content Harmony‘s Marketing Director, Kane Jamison, who spoke about “Current Internet Marketing Trends & How They Will Affect Your Organization”. He actually posted most of his content over at his blog, which is well worth checking out. It was full of useful information about digital marketing and how it is changing. I really liked his data-driven format, with lots of interesting data points to hang his presentation from.

One of the most interesting things Jamison mentioned (which he doesn’t seem to cover in his blog post) is how the change to more HTTPS Google searches has effected website owners. Links from Google search using HTTP includes search keyword information, which helps website owners tune their SEO approach (“gee, lots of people are coming here looking for XYZ, we can write more about that”). However links from searches using HTTPS do not include this data, which is a reasonable privacy and security protection. Now that Google is moving people to use HTTPS more often (due to more authenticated usage to support Google Plus and other services), this means fewer searches from Google include search keyword information. That is impacting how website owners approach SEO.

Next we heard from Josh Dirks, Founder and CEO of Project Bionic. He started by telling us that he was the grandson of an auctioneer and son of a preacher, and he had a fun speaking style to prove it. His talk was titled “Welcome to Your Social Nervous System”.

Dirks argued that social media is not about marketing at all but rather about customer service and community building. He illustrated this by relating today’s big data and social media revolution back to the days of rural villages, before the industrial revolution. In those days most folks lived in a village of 50 to 300 people and had few secrets. According to Dirk, businesses had to pay close attention to their customers and provide excellent products and services, because there was no escaping their reputation in such a small world. In the industrial revolution this all changed. Huge factories hundreds of miles away mass produced products, and there was no mechanism for them to receive feedback other than the one dimensional channel of communication called revenues. Media was all one-way: Newspapers, TV and Radio. Then the internet and social media came along and re-connected customers and companies. Now word about your business travels fast on Twitter, Facebook, Yelp, and so on. Businesses that listen and interact with customers online in a dialog are going to succeed in this new era, while those locked in the one-way world of the past will fall behind.

There was also a third speaker scheduled, but they couldn’t make it due to illness. Fortunately, Dirk and Jamison provided us plenty to discuss and think about.