Summary

June highlights from the Amundsen community:

  • How Convoy got 80% of its tech org to use Amundsen

All that and more details below!

Check out last month’s highlights here.

Don’t forget: Join our Slack community at slack.amundsen.io. We can’t wait to meet you!

How Convoy got 80% of its tech org to use Amundsen

Video…


Learnings from dozens of companies on how to make your data catalog successful

PHOTO BY JOSHUA SORTINO ON UNSPLASH

There are only 2 goals that matter when it comes to measuring the success of a data catalog: 1) adoption, and 2) customer satisfaction. If you nail these two, you are successful.

I’m the co-creator of the leading open-source data catalog, Amundsen, which is used by 35+ companies including Instacart, Square, Brex, Asana, and many more. In this post, I share key learnings from Lyft, other Amundsen adopters, and Stemma customers on what makes a data catalog install successful.

There are learnings that we have incorporated in Stemma, but this article captures learnings that haven’t been captured in the product…


Why Snap chose Amundsen, dbt integration, lineage graph viz, mono repo, and more!

Summary

May highlights from the Amundsen community:

  • Why Snap chose Amundsen

All that and more details below!

Check out last month’s highlights here.

Don’t forget: Join our Slack community at slack.amundsen.io. We can’t wait to meet you!

Why Snap chose Amundsen

Video: Why Snap chose Amundsen

This month, we heard from Justin Jordan, a developer on Snap’s analytics engineering team…


Today, we are excited to announce the launch of Stemma — a fully managed data catalog, powered by Amundsen, the leading open-source data catalog with the largest community and broadest adoption. We raised $4.8M in seed funding led by Sequoia to bring the power of the leading open-source data catalog to every organization.

The problem: Too much data, too little trust

Over the last decade companies have first captured more data and then made it accessible to more and more people within the company.

Everyone has access to data but few know what exists, what’s trustworthy, and how to use it.

This leads to a huge productivity loss…


April 2021 — Data lineage, AWS Neptune support, Amundsen in top 10 data OSS projects, and more!

Summary

A quick summary for April in Amundsen:

  • BYOL: Bring your own lineage! We improved data lineage adding native support for ingesting, storing and serving table and column level lineage natively in Amundsen

All that and more details below!

Don’t forget: Join our…


All recent innovation in data has taken place in two areas — helping data engineers produce data, and helping data consumers (primarily data analysts and scientists) consume that data. Data warehouses and lakes are flooding with data, but the consumers still don’t know what exists and what to trust, turning them into data swamps.

The biggest gap in data-driven organizations, however, doesn’t sit in the production or consumption of data but right between them. Data Engineers continuously report being bombarded by questions from users while striving to deliver it on time and with high quality. Analysts and Data Scientists spend…


Nelson Mandela’s definition of resentment

Say you are in a relationship and you call your partner and ask when she is coming back home, and if she is going to be hungry by the time she returns back home. She replies yes so you go and get some take out just in time for her return. Only for her to come back home super tired and say that she is not hungry any more. You haven’t eaten because you were waiting for her to come and now that you are starving, you feel resentful for your partner to have made you wait for nothing. …


This post introduces the Amundsen project — its goals and users. To read more about technical architecture, see this follow-on post.

In order to increase the productivity of data scientists and research scientists at Lyft, we developed a data discovery application built on top of a metadata engine. Code named, Amundsen (after the Norwegian explorer, Roald Amundsen), we improve the productivity of our data users by providing a search interface for data, which looks something like this:

The problem

Data in our world has grown 40x over the last 10 years — see the chart below from United Nations Economic Commission for…


In your eyes
I see sunshine.
Energy, warmth and infinity
all combine
In your eyes.

Sometimes, in your eyes
I see your past.
Running away for miles and miles
Only to come home at last.

Sometimes, in your eyes
I see the pain.
Forging a relationship with that someone
Asking yourself what you have to gain.

But get up!

’Cause in your eyes
I see your strength, I see your might.
Cutting through darkness and fright.
In your eyes,
I see the light.

In your eyes
I see fire.
I see change.
I see desire.

In your eyes
I see peace.
Calm as the ocean
Fresh as the breeze.

So, just let me stare my dear
’Cause time just flies
I see love
I see life
In your eyes.


*Read Part 1, 2 and 3 of The Grand Canyon series here, here and here.

Now I was on the trail, 2 miles away from my destination, thinking about help. Thirty minutes after I laid down, I gathered my backpack, replenished more water, drenched myself again in more water.

It was around 12:45, and I still hadn’t moved much. I knew, I wasn’t going to make my worst case estimate of 1:30pm, let alone my original estimate of 12:30pm. But it didn’t matter. I got up and started walking. The sun was unforgiving and shade was no where to be…

Mark Grover

Writer, Engineer, Poet (mark.thegrovers.ca)

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store