Learning and The Art of Statistics

I think this might be the UK version of the cover design; much more preferable than the US version.

This summer I started reading David Spiegelhalter’s book, The Art of Statistics, but never got beyond the second chapter. I also had plans to learn d3 and do a personal visualization project. So much for plans. School ended and then I got bombarded with work.

Summer. Poof. Gone.

So now that I’m back at UM, I started reading The Art of Statistics, again…from the beginning. If anyone asked me about it, I would simply say that I don’t know much about it. For most of my twenty-some working years, my answer was accepted … until now.

Should, should, should…

Should designers learn to code?

Ever since I started designing websites the question, “Should designers learn to code” has been a never-ending debate. I used to think, Nah, leave it to the experts. Focus on your strengths—design! It was the easy answer; the one I, as well as many of my designer friends wanted to hear. Yet now, as a student in a STEAM program, I can say with certainty that a designer who can code has superpowers.

I’m trying to use a method of highlighting where yellow seems important and green are terms I need to learn and look up more if I don’t understand.

Should designers learn statistics?

I don’t think there is a “Pass Go” card for statistics if you want to practice data visualization ethically, truthfully, professionally. Of course you don’t need to be a statistician but as I’m learning, it sure helps you understand data. It helps you ask questions and more questions. I am convinced more than ever that as we move into this Fourth Industrial Revolution, citizens must have at least some amount of data literacy. Spiegelhalter says:

More data means that we need to be even more aware of what the evidence is actually worth.

David Spiegelhalter, Author, The Art of Statistics: Learning from Data

So yes, designers should learn statistics and The Art of Statistics is a great way to start. The format is wonderful because Spiegelhalter asks a question at the beginning of each “lesson”. He weaves jargon, concepts, history and process into stories from data which are stories about people and communities. What’s not to love? His writing isn’t dry. It is conversational, sometimes funny and always approachable. It is void of academic uppity.

Learning about distribution and logarithmic scale.

I decided to try a different approach to learning based on another book I’m also reading, Make it Stick. It’s a great book for teachers and students about learning. One takeaway was to break down anything you are learning into a series of steps or connections.

An excellent book for students and teachers.

That may seem obvious but apparently I went through life with with a different method for learning (memorization). With that in mind, I downloaded an upgrade to MindNode to try it out again (Thank you Qinyu for the reminder) to see if this process of creating a structure, a network, a series of relationships would help lodge some of these new terms and concepts into my long term memory. I’m also hoping this process makes learning statistics more enjoyable. So far, so good. (yay, me)

If you’re curious, you can click on the image or this link to see a better view of what I’ve started.

I’ve created mind maps on paper but what I like about MindNode so far is that I can move networks around easy peasy. The more I read, the more I process, I can go back and adjust as things become more clear.

Networks, Relationships, Connections

As a side note, I’m also learning how to design for artificial intelligence (or is it with?) and what has been interesting these past few weeks are how statistics, psychology, data visualization, artificial intelligence, and human-centered design are moving together and intersecting at various points as I process everything I am learning. The work I do (currently literature reviews) in the UX Lab continues to expose me to related terms and concepts.

As I move through The Art of Statistics, I’m getting early hints that I’ll be learning about analyzing performance for machine learning (regression models, algorithms, prediction). Cool. And recently, I learned about the theory of Connectivism.

There’s a path developing. Lightbulb moments.

We Lost Three Billion Birds

My husband and I have binoculars and a bird guide on a window ledge in our dining room. Sounds dorky, right? Well, I’m old enough to not care much about being a dork these days so onward.

We have a few bird feeders in our backyard and if I was home about this time of year and through the winter, we would be watching for birds and trying to identify them. Sometimes we would use our trusty guide book or use Cornell’s Merlin Bird ID app. There was also a time when we would send information about which birds showed up in our backyard.

So when I came upon this story in the New York Times about the loss of 3 billion birds since 1970 I was shocked, then deeply saddened, then alarmed.

Now, when I say shocked, I wasn’t surprised that there was a population loss. Climate change, pollution, deforestation, agricultural practices, invasive species … the list goes on. But really … danger, Will Robinson!

3 billion birds since 1970.

3 billion birds. 50 years. Gone.

Now, this graphic is beautiful. The colors, the structure, the information is clear and helpful. This is the online version in the Science Section:

Birds Are Vanishing From North America – The New York Times

Clearly, hit hardest are the Grasslands. The article mentions that even Robins and Sparrows, and Starlings (those invasive species) experienced significant declines.

Robins and sparrows.

a full-blown crisis.

David Yarnold, president and chief executive of the National Audubon Society

Isn’t everyone shocked and alarmed?

Can you imagine a world without birds? I don’t know about you but that gives me the willies. In fact I recall a moment while walking in our neighborhood when I realized there were no bird sounds. It was the eeriest feeling. I went out every morning listening specifically for a chirp or a call. What a relief when I finally heard one.

Conjuring a world without birds is a thing I don’t dare imagine, like the death of a child. Their fate is our own.

Joel Sartore Photographer

Can you picture 3 billion birds?

So, while writing this post, I realized that I wanted to show you what losing 3 billion birds looks like. Can you picture it? I can’t. If I had to explain to my 7-year-old niece what a loss of 3 billion birds looks like what would I show her?

I couldn’t find anything but this piece, Drowning in plastic: Visualising the world’s addiction to plastic bottles by Reuters came to mind.

It is insane to see the relationship of plastic bottles to iconic structures like the Eiffel Tower.

1.3 billion bottles. “Every day the equivalent of a bottle pile half the size of the Eiffel Tower in Paris is sold around the world.” Reuters

Would the story have greater impact and meaning among non-bird experts and fans if there was a way to see scale; a tasteful rendering and relationship of the number of birds to something familiar like the Statue of Liberty or covering an area the size of Texas or ?

I’m thinking yes. Do I have an exact solution? No, but I’d love to explore it some time in the future. In the meantime, I’m going to continue to help conservation efforts and take every day I see birds (even those pesky Starlings) as a gift.

#gratitude

Project Pitch: The Poetry of Anne Sexton

Today we pitch our project proposals.

Time limit: 3 minutes.

Writing poems, my quiet place.

I signed up for a poetry class during summer break while an undergrad at The University of the Arts (UARTs) pursuing my BFA in Photography. I had no idea what I was getting into but I went with it. The experience turned out to be one of the unexpected “bests” of my life. That summer was also the year I learned, in great detail what happened to my grandfather , a man I never knew because someone decided to take his life long before I was ever imagined.

Writing poems gave my anger and loss (can you mourn someone you never knew?) a place to rest. It was also the first time I recited a poem in public. Granted, it was a room of my classmates but for someone who honestly prefers to sit in the audience, this was big. Huge.

Transformations was my introduction to Anne Sexton.

It’s been awhile since I’ve read her poems and this idea of a project has been a wonderful way to reacquaint myself with her and her work. It’s been, well, decades. Does her work resonate with me now as it did then? I believe this project will be an emotional journey as much as it is a learning experience. What will I discover?

I’m proposing a visualization that slightly petrifies me.

I tweeted last night that I’m already in the “dark swamp of despair”.

Self-learning: Python, D3, scraping data, build a corpus… oh my.

Blogs, tweets, StackOverflow, GitHub, Codecademy, Lynda … you name it, I’m searching for “How to …” often.

  • Learning python: New jargon. Decisions about IDEs. Pandas… they aren’t cute black and white furry animals?
  • Learning how to properly analyze and clean text data: Pre-processing? Does my data need to be organized in a csv file or like a massive dump in a txt file? Looking for tutorials on how to prep text with Jupyter notebook) Soooo many questions.
  • Learning D3 is not going to be easy though I am hopeful Amelia Wattenberger’s book, Full Stack D3 and Data Visualization will be a huge help. (She is based in Rochester, NY – cool.)

Some classmates have suggested that I use R instead of Python. I’ve gone back and forth on this. Maybe I’m crazy but I prefer to learn Python because it seems a language that crosses many disciplines for many applications. Then, there’s the fact that python was named in honor of Monty Python’s Flying Circus!

Part of this is also recognizing that I know what is good but my abilities to execute are far below what it probably takes to meet my own expectations. Ira Glass has something to say about this and I’m trying to find some comfort. One would think familiarity of this mind space would make it easier each time. But no…

Still, I have some bright sides.

Mindy McAdams, a data journalism professor at the University of Florida was helpful in getting me setup with Python. I didn’t know it at the time but miniconda was the right way to go and given the number of IDE options, I’m so glad I went with Jupyter Notebook. (Seriously Mindy I cannot thank you enough.)

The bright side wouldn’t be complete without mad props to Lenny Martinez, one of UM Interactive Media’s data journalism professors. His positivity is all goodness when I feel like I’m drowning.

And, to end on a positive note, I do know how to install packages and I know kernals and cells, virtual environments and I’m not afraid of Terminal so much anymore. Not bad. Perhaps I need to just get over the fear that I’ll break something if I type incorrect syntax and also not worry so much about creating a properly formatted text file.

Ah, the “known knowns, known unknowns, and unknown unknowns”.

Notes from the Classroom: Three Data Visualization Experts

It’s always inspiring to hear from people who love what they do. Our Data Visualization Studio class was fortunate to welcome three (remote) guest speakers: Luís Melgar, Julia Wolfe and Maarten Lambrechts. I want to share my notes and takeaways both as a reminder for myself and to share with anyone learning more about data visualization like me. Warning: This is a long post.

Luís Melgar: learn something new & a great way to Quiet Imposter Syndrome

Profile at Guns in America
Twitter

What I particularly loved about Luís’s talk was the mix of professional and personal advice. It was positive and endearing, a talk that makes you feel good and ok exactly where you are.

Keep a data journal

  • Log your data, track your sources to maintain sanity. Truth, this. If you’ve ever felt like you were drowning in data or hunting for sources in the middle or near the end of a project, this is your life jacket.
  • Write down what you did with the data, how you came up with your conclusions. This is accountability.
  • Track your path to the very end.

Data is imperfect

I love how Luís phrased this tip: interview your data.

Questions to ask:

  • Where is it coming from?
  • What form?
  • How it is defined?
  • What questions does it bring to mind?
  • How was the data collected?
  • Who collected the data?

Get clarity. What can you really see from the data? Dig into it.

Advice about coding

  • Is your code reproducible?
  • Make it easy to track mistakes
  • Comment your code
  • Use another language to double-check your calculations
  • Test, test, test
  • Review, review, review
  • Check, check, check.
  • Rinse and repeat.

Learn something new

Pick a project, set a goal. Do it.

Expect the unexpected

Some people may give you data in a PNG. No, really.


Below: Two graphs from Left In Carry-Ons, TSA Agents Find Thousands Of Guns At Security Checkpoints by Luís Melgar.

Surprising.
Why?

Make peace with yourself

Coding is hard. So, for those who aren’t natural coders, it’s OK if you can’t execute what you have in your mind. Luís shared his experience working on his Master’s capstone and some of the hurdles he came across and how he learned to adapt and work around his own limitations.

Know your technical limitations

Different companies and organizations use a wide range of publishing tools. Understand what is possible within those constraints.

Always keep your audience in mind

Make your visualizations responsive and usable on mobile. This is a must.

Don’t ask for too much advice.

This tip made me think of some photographers I know who seem to ask too many people to give them feedback about their work. Let me clarify: I think it is OK to ask as many people as you want but take advice from just a few— the people you trust most. It was nice to hear Luis same the same. Basically everyone has an opinion but whom do you trust?

Love your Classmates

Yes indeed. Luís shared a funny photo of some of his classmates (with some random dude cute out – lol) and how he continues to stay in touch. He shared a couple of memories about music and food which was sweet and funny.

Life is about relationships. The relationships you build or burn in graduate school will have a big impact on your life moving forward. Take the time to learn something new about the people you see every day. They will enrich your life in so many ways.

Keep a notebook of success stories

This. I was surprised to hear him talk about imposter syndrome and it was refreshing to hear someone with so much more experience share that even he suffers from it from time-to-time. He shared how women and minorities tend to experience it more. Interesting.

His solution? Keep a notebook close by where you write down your success stories —what you have learned or accomplished. Big or small. When you are feeling like an imposter, or perhaps just feeling a bit low or unsure, refer to your notebook. It’ll make you feel better; that you are making progress; that you are learning; that you are exactly where you need to be and where you are supposed to be.

Thank you Luís. It was wonderful and inspiring to hear from a graduate of our program.

Julia Wolfe: Data Journalism Needs You

Profile at FiveThirtyEight
Tweet, Tweet

If you are journalist with technical skills, the outlook is good.

Julia Wolfe, Data Journalist, FiveThirtyEight

Julia’s talk made me think about why I have such an interest in data visualization. She shared what she loves about it and I had to agree that her reasons for being a data journalist are compelling:

  • The day-to-day variety
  • Learn something new often
  • The work you do is for public good. You can help clear up misinformation and help explain why [x} affects daily life.
  • Telling stories

She also gave us some insight about what makes working at FiveThirtyEight different from working at large newspapers and also highlighted a few other places that are doing great data journalism:


A detail from, What First-Quarter Fundraising Can Tell Us About 2020 by Julia Wolfe.

One of my favorite visualizations about the Democratic candidates. The animation is such a nice touch.

Data, Design & Code Advice

Her advice aligns with much of what Alberto has shared as well as what I’ve read in books and online. It helps to hear similar advice but in a different way.

  • Analyze data: Find the story. What are your questions?
  • Be skeptical of the data
  • Talk with a subject matter expert or know enough about the subject to either spot something surprising or identify a problem.
  • Understand basic design sensibilities: Color, hierarchy, white space, alignment, typography.
  • Edit and proof: Layer your workflow and establish a process for self-editing.
  • Comment your code so you can remember it later. Save every step.
  • Keep your data organized
  • Do rigorous spot checking
  • Make sure your source aligns with your visualizations
  • Check your outliers. Are they reflected in the source?
  • Provide documentation. As you work, make notes or comments about what you found and how. You may need it to back up your work, your conclusions.
  • Bring context to data and moments.

Skills

  • Learn how to get and scrape data. What kind of unique data can you get; something that no one else has?
  • Freedom of Information Act (FOIA). Luís Melgar mentioned this as well. Understand how to file a FOIA request. (See links below)
  • Excel. Ramp up your skills.
  • Reporting. Hit the streets if needed.
  • Interview. Yes, talk to people!

Knowledge

  • Understand state laws. What are you legally within right to request and what are institutions required to give you? I loved this tip. I learned state laws and policy is important secondary research for product design, too.
  • Understand different data formats: PDF? Excel? I took this to also mean, understand how to read them and how to extract data from PDFs (Tabula to the rescue …)

Conferences to Attend

Twitter is a must

Enough said.

More Resources

Julia provided us with some great resources (note: I’m not 100% sure about all these links but if they aren’t exactly what she had in mind, I hope they come close):

Thanks Julia for all the great tips and speaking with us so honestly. You are an inspiration.

I’m excited and eager to hear from more women in data viz.

Maarten Lambrechts

Website
Tweet, Tweet

Maarten walked us through his process for Why Budapest, Warsaw, and Lithuania split themselves in two published on The Pudding. It was fascinating to hear about how he was first introduced to the story and how he worked with the team at The Pudding to get it published. Takeaway: if your pitch to one person or place doesn’t get accepted, keep trying.

I’m glad he did because he made a story I would probably pass over into a story I wanted to read. Even more, I wished for a follow-up. What more could I learn? For example, when he shared a brief look at his sketches and a sentiment analysis he had done, I wanted to learn more. What patterns are developing in Europe?


Details from Why Budapest, Warsaw, and Lithuania Split Themselves into Two

This is where things start to get interesting.
Love this transition and alternate view of looking at the same data.

Honestly, Maarten’s talk was a good inside look at what it takes to be a data journalist/ data visualization designer. It was a bit of a wake up call, too. It takes a certain level of tenacity, a doggedness, if you will. Perhaps this is why I latched on to a quote he made during an interview with Open Belgium 2016:

If you want to become a data journalist, you should start thinking like a programmer.

Maarten Lambrechts

There’s just something about programming that is empowering. Perhaps because it forces you to look at a problem in many different ways. It changes how you think about things, look and interact with the world. Hmm…

Like many other professions, what most people see is the finished, polished work. Viewers/readers don’t see what goes on behind the curtain. That’s what I enjoyed most about Maarten’s talk. He showed us his sketches, his thinking; what worked, what didn’t. It was a comfort to see the mess.

Thank you Maarten for the incredible resources and sharing your story with us. Consider me a new fan of your blog.

Thanks

Wow, you made it to the bottom. Nice.

Thank you data visualization community for your generosity. Ya’ll are icing on the cupcake.

Thank you, Shirley Wu

Two mailing addresses, a few missed issues of Net Magazine.

Missing magazines show up—yay!—but left unopened for weeks.

Finally open the envelope, glance at the covers and they sit for another week. I was beginning to feel that read-your-New-Yorker-magazine-pressure every time I passed them.

But just yesterday when I sat down to write a blog post about a yield curve I saw (saving that for later), I noticed a small cover line on the August 2019 issue of Net Magazine.

Learn from a Data-Viz Whizz

Really? Cool.

Who is the Data-Viz Whizz?

Flip, flip, flipShirley Wu.

Opening spread in the Voices section of Net Mag, August 2019.

Shirley Wu, one half of the popular Data Sketches project, creates highly interactive, beautiful data visualizations. Here, she gives us a look behind the scenes and shares the lessons she’s learned.

Intrigued, I immediately sat down to read about Shirley Wu.

It was an inspiring read. I love stories about people who make changes; that she switched from being a front-end software engineer to become a data visualization designer is cupcake.

So, do you recall periods in your life when you are planning your next step or struggling or hoping for something but not sure what and the universe sends you a person or a moment or a sign to help you take that one small step forward?

This interview with Shirley is one of those moments. Noticing that small cover line near the UPC symbol is nearly impossible but I did and that led me to discovering Beautiful for the first time. Why is this important? Because I’m planning to tackle my first interactive data visualization of poems (Pablo Neruda? Anne Sexton? Maya Angelou? — I need to decide) and it helps to see what other data viz designers have done using text. Part of learning is seeing and understanding what is possible. It may take me some time to reach Shirley’s level of talent but her work and her words were a spark.

Correction: In my excitement, I had flipped Shirley Wu and Nadieh Bremmer’s work in my mind. My sincere apologies to both women. Nadieh is the designer behind Beautiful and Shirley is the designer of Explore Adventure (below), an equally beautiful (see what I did there?) and fun visualization about the travel search connections between countries, seasons, attractions, and more.

Screengrabs from Explore Adventure. This section about searches for Qin Shi Huang was the most interesting; however, I’m trying to still understand how I’m supposed to know the searches happen during spring, summer, winter or fall when just looking at the visualizations.

Lesson learned? Give yourself enough time to triple check your work, what you read, and own up to your mistakes.

Beautiful visualization interaction
The snaking animation is delightful and on hover, I learned more.

What I love about Beautiful is the overall simplicity, the subtle animations and the surprising level of detail and information which isn’t obvious at first. It’s fun and interesting. My only wish: a little more feedback during my interaction with the top 10 words per language.

Beautiful visualization
I wanted a slight animated fade or subtle change in value when I hovered over any of the shapes for each country.

Legends is just stunning and ok, I admit, I have a thing for that color palette. The cool factor is huge. Immediately I wished for Legends to be realized into a physical space that I could walk in and around (VR anyone?) with additional layers of information as I interact with each crystal.

Legends visualization from above
This feels like you are flying just above the surface of an alien planet (I watch a lot of Sci-Fi).
Legends visualization side view
I imagined myself walking among these crystals. What if I could touch them and they would light up and reveal something interesting?

I also admire the fact that she shares her knowledge about D3.js with others through workshops (with live coding!), user groups and online courses. She mentions her process and a lot of the tools she uses to clean, understand, explore, prototype, and design. It is a list I’m definitely planning to check out as I begin my journey learning D3.js through Coursera and making my first interactive visualizations.

So, thank you, Shirley Wu for making feel even more excited about data visualization, sharing lessons you’ve learned, your take on tools, and showing me what is possible with D3.js.

I cannot wait to see more of what you create.

PS: It’s always nice to read about people you know, especially when you are also learning from the same. She mentions Alberto twice. The first when he invited her to dive into data that would result in Beautiful and the second when she mentions how teaching forced her to learn so she read a few books. The Functional Art, she says, is “one of my favourites”. Cool.