Saturday, November 12, 2016

NYC, Population and Density

Take a quick look at this viz:


What the heck is up with Manhattan? Too much time at the Shake Shack? And why does Staten Island look like a deflated balloon? The above map is what is known as a cartogram. A cartogram takes a feature within a geography and resizes it by that feature. In this picture we see the 5 Burroughs of New York, divided into their separate neighborhoods, and sized by population. New York City's normal geographic boundaries have their area resized to match their relative populations. I have also colored the neighborhoods to further show the largest neighborhoods by population.
By contrast, we can see how the geography has been distorted when we overlay the actual boundaries.
This type of cartogram can come off as too literal however; the distortions sometimes do not offer enough information for the mind to really better understand area. In a broad sense, we get that Manhattan is full of people and Staten Island has very few people. But how many fewer?
The eye is terrible at reading absolute area and making decisions from it. So a simple bar chart, while somewhat dry, is the easiest way to understand how New York City is divided.
We can see that Brooklyn easily bests Queens and Manhattan for population, while Staten island has roughly the same population as the Reno-Tahoe area. But even that diminishes Manhattan. Manhattan's density is on par with some of the densest cities in the world. The large populations of Brooklyn and Queens are still pretty spread out.
Instead, let's look at each Burrough's density:
The labels show the percent of total density relative to Manhattan while the bars show, in thousands, the number of people per square mile. In other words, Staten Island is less than 1/10th as dense as Manhattan. The large total area of Queens means that while it is the second largest of the five Burroughs, its citizens can stretch their legs out comfortably without fear of bumping into another human.
Ultimately, we would be best off giving people a chance to actually explore the data a bit. What are the populations of neighborhoods in New York City by neighborhood? How about if we sort it by Burrough? Take a look below and see how New York's neighborhoods stack up.

There Are Always Octopuses

"I don't know sir, everything was going fine then we lost control" the technician looked at her row of gauges and tapped on one. It didn't move, not that tapping on it was supposed to. 

"Well what do you think? We've already restarted the project six times. I'm not really willing to wait for another opportunity like this," the supervisor scratched his peppered beard. A crumb from lunch fell onto the gauge marked mean temperature. "Last time it was 300--" 

"Yeah, 300. I've read the report, better than the first one though." 

"Maybe, or maybe not, honestly we ran out of patience on the last one and cut it short." 

The technician paused. Her job was strictly to monitor the gauges and she hadn't been there long. She liked her supervisor but didn't know quite what to make of him yet. "Sir, honestly, it's your call. Everything is setup should we need to have an emergency shut down." 

The supervisor was restless, this assignment had been a once in an eon opportunity. There weren't many assignments with this level of complexity or responsibility. But he sensed that the assignment was also a Sisyphean one. His last manager had hated how he handled the Andromeda merger and he distinctly recalled the words that had led him to this promising backwater. "I think your skills are better suited to a more nuanced role. I put you in a position where you were away from what you loved, and that was my fault. I want to get you back to where you thrive. We have an opening at our Orion arm and I have recommended you for an R&D project. It's had some fits and starts but we think you can get it moving."  

It sounded like such responsibility at the time, resurrect a nearly extinct project, bring a revolutionary and heretofore unknown product to bear. Sign me up, he thought. But then he got there and saw how much disarray everything was in. They had lost an entire lab, and the remaining one was just wrecked. It was a scorched Earth, just everything in shambles, and little sign that any actual progress had been made.  

His first task had been to get the project organized, and disavow the actions of his predecessor--like any good supervisor. There was one good thing from the remnant, a couple of product lines had all the necessary characteristics to eventually flourish. It was just a matter of putting the right circumstances together. Other than that it was an utter crap shoot. 

I mean, let's just start with the mosquitoes. Those little bugs ruined four good batches and they weren't even the worst contamination. The last guy had somehow managed to inject viruses into the environment and lost, by conservative estimates, several thousand viable batches. 

And the lack of accountability was awful too. The team had somehow managed to never accurately account for anything, leaving the entire project with a huge question mark. Was it even doing anything? 

The supervisor turned to his tech. She was smart, young, and ready to prove herself. She would go far with her natural instincts toward what would work. 

But today was not either of their days. In what seemed a blink of an eye they went from a strong controlled scenario where everything was going perfectly to half their gauges going red. What had they missed? 

"Sir, my thesis tackled these kinds of scenarios. This isn't atypical, but it is more extreme than I had expected. We both missed it, but I don't think it's ever happened like this. Usually the system alerts us sooner. I can check the wires to see if an indicator is broken, but I'm pretty sure we're just dealing with an outlier." The technician felt guilty, she was confident that her first time would be flawless, but it had already proven anything but. The first full cycle on the job had been a mess, first by turning the thermostat down too much and then by accidentally cranking it back up too fast. They had lost one of the candidates but the other one actually thrived. A lucky break.  

But now this. After exploding out the gate, a series of self inflicted losses had made activating the cultures a nearly impossible task. Well, there were always octopuses, she would mumble to herself.  

"Sir, my suggestion is a targeted loss. We still have at least four other candidates that could take over." 

"Don't be absurd. We can't just trim the tree at this stage, two steps back that would have been ok, but you know that calling it would cause a total reset. We wouldn't have our subjects nor would we have candidates." The supervisor stretched his neck and sighed. "Look, what options do we have?" 

"We could introduce a stable subject? But that could go all sorts of ways. I'd estimate total losses anyways in most scenarios." 

The supervisor looked at the gauges. Hundreds of dials and beeping lights, indicators and graphs, stretching across the entire room. They all told a story but none told him what to do.  

He paced for a few seconds pretending to be lost in thought. He thought of nothing. Screw it. 

"We are this far in. There is still a chance it could work right?"  

"Not much, but if we leave it and see where it goes we always have octopuses at least."  

The supervisor laughed, "I wish! But that's a good one. We always have octopuses. I like your optimism." The supervisor took a deep breath in, "ok, let's leave the humans and hope they don't kill themselves. It's better than waiting another 65 for something sentient to show up." 

"Alright sir, but I'll keep a few meteors around just in case. What should I put in my report to the rest of the team?" 

The supervisor paused.

"How about that we are holding on another mass extinction because we see real promise in octopuses." 

Wednesday, February 3, 2016

Diversity in Technology

I recently listened to a compelling article on NPR about the diversity of the technology industry.

Intel Discloses Diversity Data, Challenges Tech Industry To Follow Suit
http://www.npr.org/sections/alltechconsidered/2016/02/03/465270938/intel-discloses-diversity-data-challenges-tech-industry-to-follow-suit 

The article website has some great tables that illustrate the big gaps in hiring practices, but it's hard to see how different 1% is from 30%. So I built a viz that should cut to the quick a bit more.

What are your thoughts when looking at this data?