Image for post
Image for post

All recent innovation in data has taken place in two areas — helping data engineers produce data, and helping data consumers (primarily data analysts and scientists) consume that data. Data warehouses and lakes are flooding with data, but the consumers still don’t know what exists and what to trust, turning them into data swamps.

The biggest gap in data-driven organizations, however, doesn’t sit in the production or consumption of data but right between them. Data Engineers continuously report being bombarded by questions from users while striving to deliver it on time and with high quality. Analysts and Data Scientists spend…


Image for post
Image for post
Nelson Mandela’s definition of resentment

Say you are in a relationship and you call your partner and ask when she is coming back home, and if she is going to be hungry by the time she returns back home. She replies yes so you go and get some take out just in time for her return. Only for her to come back home super tired and say that she is not hungry any more. You haven’t eaten because you were waiting for her to come and now that you are starving, you feel resentful for your partner to have made you wait for nothing. …


This post introduces the Amundsen project — its goals and users. To read more about technical architecture, see this follow-on post.

In order to increase the productivity of data scientists and research scientists at Lyft, we developed a data discovery application built on top of a metadata engine. Code named, Amundsen (after the Norwegian explorer, Roald Amundsen), we improve the productivity of our data users by providing a search interface for data, which looks something like this:

Image for post
Image for post

The problem

Data in our world has grown 40x over the last 10 years — see the chart below from United Nations Economic Commission for…


In your eyes
I see sunshine.
Energy, warmth and infinity
all combine
In your eyes.

Sometimes, in your eyes
I see your past.
Running away for miles and miles
Only to come home at last.

Sometimes, in your eyes
I see the pain.
Forging a relationship with that someone
Asking yourself what you have to gain.

But get up!

’Cause in your eyes
I see your strength, I see your might.
Cutting through darkness and fright.
In your eyes,
I see the light.

In your eyes
I see fire.
I see change.
I see desire.

In your eyes
I see peace.
Calm as the ocean
Fresh as the breeze.

So, just let me stare my dear
’Cause time just flies
I see love
I see life
In your eyes.


*Read Part 1, 2 and 3 of The Grand Canyon series here, here and here.

Now I was on the trail, 2 miles away from my destination, thinking about help. Thirty minutes after I laid down, I gathered my backpack, replenished more water, drenched myself again in more water.

It was around 12:45, and I still hadn’t moved much. I knew, I wasn’t going to make my worst case estimate of 1:30pm, let alone my original estimate of 12:30pm. But it didn’t matter. I got up and started walking. The sun was unforgiving and shade was no where to be…


*Read Part 1 and Part 2 of The Grand Canyon series here and here.

I thought about help. I wondered if I should ask for it, or if I could make it on my own. I wondered if there are two types of people in this world — those who are very individualistic and independent and those who believe in giving help and are ok with receiving it too.

I thought about the night before when I walked into the park without having a place to sleep and the help I needed to find a place. I had gotten to…


*Read Part 1 of The Grand Canyon series here.

As I approached the Grand Canyon, it started to pour. That along with Imran’s suggestion and a few hours of subconscious thought had made the decision for me loud and clear — it was a terrible idea to hike down the Grand Canyon at night. So, I was going to be sleeping at the rim that night and hiking whatever I could the next day. But, I still didn’t have a place to sleep. …


I hadn’t been this tired in a while. I found some shade under a big rock, threw my backpack down and lay dead and flat on the dusty Bright Angel Trail, like nothing else mattered. I was only two miles away from my destination — the Bright Angel Trailhead on the South Rim — but it seemed eons away.

People passing by would ask if I was doing ok, to which I would always lie — ‘Yes, I am’. Always answering a proud ‘Yes, I am ok’ whether or not that was true. Similar to real life, I suppose.

A…


I realized as I ate makki ki roti today, that I was eating was an Indian arepa. And, that kheer was just an Indian arroz con leche and that a deep fried pierogi was a Polish samosa.

It’s interesting how different cultures evolved through the same patterns, and to very similar outcomes, sometimes in collaboration but many times in isolation. How cultures evolved to feed themselves the same way, raise their livestock, build communities and families the same way.

And, today, we all travel. We all travel to see the different cultures and learn more about those differences. But, the more we travel, the more we find, not how different, but how similar we all are.


El Camino de Santiago is a set of ancient historic paths, a pilgrimage people have made over centuries to visit the Cathedral of Santiago. All caminos lead to the city of Santiago de Compostela in northern Spain. The longest and most popular of all caminos is Camino Frances (the French way), starting in St. Jean Pied de Port, in southern France, ending all the way in Santiago de Compostela, over 780 kilometres away.

Image for post
Image for post
A map of original Camino Frances

Recently, I undertook a portion of El Camino Frances, walking about 110 km from the city of Sarria, in Lugo to Santiago de Compostela. And, deciding along…

Mark Grover

Writer, Engineer, Poet (mark.thegrovers.ca)

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store