DataHub v0.8.7

Release Stability

• There are a few bugs reported on this release that are fixed in 0.8.8. Users are highly recommended to skip past this release!

Release Highlights

• Dataset Profiling and support for time-series metadata
• UI for ML Models, Features; support for AWS SageMaker and Feast
• Cli: support for rollback operations after ingestion
• Integration fixes for Looker, dbt, and many more.
• Demos for all these features are available in our July Townhall video

ChangeLog

• #3021 @kevinhu feat(ingest): extract dbt versions into custom properties
• #3020 @gabe-lyons fix(caching): refetch query on update
• #3019 @kevinhu fix(ingest): don't assume Glue job description always exists
• #3000 @topwebtek7 fix(react): fix weird 0 rendering possible bugs
• #3018 @dexter-mh-lee feat(ingest): add kafka emitters for MetadataChangeProposal format
• #2999 @jjoyce0510 fix(gms): Adding Rest.li Write-Time Model Validation
• #3009 @jjoyce0510 fix(quickstart): Bumping Default Memory for GMS and Frontend
• #3007 @jjoyce0510 fix(gms): better logging on failed MCL / MAE
• #3008 @gabe-lyons fix(blank pages): removing apollo caching
• #3006 @jjoyce0510 fix(ci): using AspectExtractor instead of removed SnapshotToAspectMap
• #2998 @gabe-lyons fix(graphql): fetching data platforms using standard procedure
• #2944 @EnricoMi refactor(test): Refactor GraphService tests
• #2972 @jameslamb fix(ingest): map all LookML dimension types to corresponding avro types
• #3005 @dexter-mh-lee fix(ingestion): Safeguard against empty values for profile ingestion
• #3002 @dexter-mh-lee fix(datahub-upgrade) add config registry to datahub upgrade container
• #3003 @jjoyce0510 fix(dataset stats): Fix checks for existence of row and column counts
• #2997 @topwebtek7 feat(react): update dataset documents tab with a merged document column
• #2991 @topwebtek7 feat(react): update search result has result counts for each entities that has result
• #2983 @jjoyce0510 Introducing TimeSeries Aspects + Dataset Profile (Stats) Aspect
• #2984 @dexter-mh-lee fix(browse): Fix browse pagination and multi-browse path issue
• #2995 @aseembansal-gogo docs(ingest): Add instructions to install required dependency
• #2960 @gabe-lyons feat(deletes): add run commands (list, show, rollback) to datahub ingest
• #2994 @chinmay-bhat docs(ingest): fixed Snowflake recipe to escape dollar-sign
• #2981 @hsheth2 docs: remove a few outdated docs
• #2988 @jjoyce0510 docs: add docs on extracting container logs
• #2963 @hsheth2 test(ingestion): run full tests on both python versions
• #2967 @jameslamb fix(ingest): add more debug logging to LookML metadata ingestion
• #2966 @jameslamb fix(ingest): ensure that LookML files are always parsed in the same order
• #2965 @jameslamb fix(ingest): ensure workunits are created for all LookML views
• #2982 @gabe-lyons fix(tags): fixing tag applied to module for tags w/ colons in the name
• #2961 @gabe-lyons feat(ml-model): adding ml models and ml model groups
• #2975 @kevinhu feat(ingest): type stubs for boto3
• #2979 @jameslamb perf(ingest): remove unused variable in Looker ingestion
• #2980 @hsheth2 fix(ingest): infer bigquery project identifier
• #2978 @chinmay-bhat fix(ingest): fix hive ingestion to respect database configuration
• #2976 @hsheth2 feat(ingest): stricter deserialization for MCE JSONs
• #2959 @kevinhu feat(docs): tutorial for writing a custom transformer
• #2977 @hsheth2 fix(ingestion): isolate dependency requirements of airflow hooks
• #2962 @hsheth2 feat(ingest): add timezone validation to bigquery usage
• #2974 @dexter-mh-lee fix(elasticsearch-setup): fix elasticsearch setup for aws
• #2952 @hsheth2 text(ingestion): test multiple python versions in CI
• #2958 @hsheth2 feat(ingest): add Airflow TaskFlow example
• #2950 @kevinhu fix(ingest): patch lookml types and refactor ingestion sources layout
• #2957 @jameslamb fix(ingest): match nested LookML files mentioned in 'include' statements
• #2956 @gabe-lyons Revert "fix(gql): removing data platform caching in gql (#2947)"
• #2955 @kevinhu feat(ingest): ingest descriptions from dbt models
• #2948 @hsheth2 fix(ingestion): add more mypy annotations
• #2946 @hsheth2 feat(ingestion): test GMS connections before ingestion
• #2947 @gabe-lyons fix(gql): removing data platform caching in gql
• #2949 @hsheth2 test(ingestion): fix flaky package discovery test
• #2951 @kevinhu feat(docs): update videos and integration logos
• #2953 @hsheth2 fix(ingestion): resolve test bugs for 3.6
• #2943 @kevinhu feat(ingest): add logo and platform entry for Glue
• #2940 @hsheth2 fix(ingest): handle quotes in lookml properly
• #2938 @kevinhu feat(models): remove versions from metrics and hyperparams
• #2942 @hsheth2 fix(ingestion): make snowflake database names lowercase
• #2939 @hsheth2 feat(ingest): use urn builders in looker and validate data platforms
• #2941 @aseembansal-gogo refactor(ingest): make code pythonic
• #2937 @kevinhu fix(ingest): allow custom Glue scripts
• #2921 @kafkahw refactor(datahub-web): removing frontend Ember app (i.e. datahub-web folder)
• #2913 @hsheth2 fix(ingest): refactor + fix recursion in lookml file loading logic
• #2925 @hsheth2 feat(ingest): improve bigquery-usage robustness and docs
• #2931 @aseembansal-gogo fix(ingest): fix workunit name to be consistent with other sources
• #2935 @kevinhu fix(ingest): fix browsepaths and ownership urns
• #2930 @aseembansal-gogo fix(ingest): glue add support for mapping varchar, decimal types
• #2929 @kevinhu feat(ingest): refactor mlModel grouping and add browsepaths
• #2934 @hsheth2 docs(ingest): update looker + docker script docs
• #2926 @hsheth2 feat(ingest): add make_data_platform_urn method to builder
• #2932 @topwebtek7 feat(react): surface edited descriptions on search preview for dataset, datajob, dataflow, chart, dashboard
• #2911 @hsheth2 fix(ingest): add quotes to secured kafka yaml config example
• #2927 @kevinhu feat(ingest): dbt aliases
• #2806 @saxo-lalrishav fix(react): enable relation between glossary term and datasets searchable
• #2910 @kevinhu feat(ingest): extract SageMaker metrics, hyperparameters, and external URLs
• #2915 @aseembansal-gogo docs: update docs for consistency in naming
• #2922 @kevinhu feat(ingest): test dbt ingestion with and without schemas
• #2924 @hsheth2 fix(ingest): note that views are not supported for Athena
• #2920 @hsheth2 feat(ingestion): support multiple project IDs in bigquery usage stats
• #2923 @hsheth2 fix(ingest): pin snowflake sqlalchemy connector
• #2909 @hsheth2 feat(ingest): add support for Oracle spatial types
• #2917 @kevinhu docs(ingest): update sample recipe and test input for dbt
• #2887 @topwebtek7 feat(mlFeatureTable): add graphql, ui/ux for mlFeatureTable, mlFeature, mlPrimaryKey entities
• #2916 @kevinhu fix(ingest): stringify all dbt custom props
• #2898 @aseembansal-gogo feat(ingest): Add option to change name of database for postgres
• #2912 @hsheth2 fix(ingest): issue a warning if the column list is empty
• #2894 @kevinhu feat(ingest): lineage for SageMaker model endpoints and groups
• #2905 @hsheth2 feat(ingest): add can_add_aspect method for MCEs
• #2906 @hsheth2 test(ingest): update tox test configurations and test airflow 2.x by default
• #2904 @jjoyce0510 fix(frontend): Don't use Apollo Cache for IsAnalyticsEnabled query.
• #2877 @remisalmon feat(ingest): use node comment as description if existing else default to key
• #2889 @hsheth2 fix(react): avoid displaying "0" for ignored timestamps
• #2890 @gabe-lyons fix(search): fixing case where someone issues a null query
• #2893 @hsheth2 fix(ingest): use logger.warning instead of logger.warn
• #2888 @jameslamb fix(ingest): change LookMLSource._get_upsteam_lineage() to _get_upstream_lineage()
• #2901 @topwebtek7 feat(react): update schema history visualizing, truncate long type, original desc bug
• #2891 @hsheth2 fix(ingest): correct globs in lookml model discovery
• #2902 @kevinhu feat(ingest): add connectivity check for Looker
• #2597 @wan54 feat(react): configure Cypress + MirageJS + GraphQL mock for functional testing plus a couple of example tests
• #2903 @shirshanka docs: update docs for July townhall
• #2900 @kevinhu fix(ingest): string-ify dbt custom props
• #2899 @jjoyce0510 fix(docs): fixing miscellaneous docs
• #2788 @saxo-lalrishav fix(glossary):default browse path for glossary term
• #2868 @kevinhu feat(ingest): extract lineage between SageMaker jobs and models
• #2884 @dexter-mh-lee fix(search): Fix index builder
• #2883 @hsheth2 docs: revamp adoption section
• #2882 @hsheth2 fix(ingest): fix druid misconfiguration bug
• #2881 @hsheth2 fix(ingest): default to unlimited query log delay in bigquery-usage
• #2790 @saxo-lalrishav fix(search): enable search on business glossary terms
• #2872 @hsheth2 build(ingest): reduce dependencies for dev install
• #2874 @topwebtek7 fix(react): fix bug in description update modal
• #2866 @hsheth2 build(ingestion): add version prompt to release script
• #2869 @kevinhu ...

DataHub v0.8.6

Release Highlights

• Fix issue when using Elasticsearch as graph database in certain configurations
• Fix caching issues in React UI
• Efficiency improvement for schema aspect storage
• Improvements and fixes to various ingestion sources

Changelog

• #2861 @gabe-lyons fix: fixing lint issue
• #2860 @gabe-lyons fix(frontend): handling null aspects in AspectType
• #2857 @kevinhu feat(docs): carousels for videos and articles
• #2855 @topwebtek7 feat(react): implement visualizing historical versions of dataset schema
• #2856 @topwebtek7 fix(react): fix searchbar width issue in small screen
• #2848 @dexter-mh-lee feat(k8s): Extract helm charts into a separate repo
• #2844 @hsheth2 fix(ingest): handle 'fields' list missing in bigquery-usage
• #2852 @hsheth2 fix(ingest): delete pycache files when running clean
• #2853 @hsheth2 docs(ingest): remove hanging sentence from docs
• #2830 @kevinhu feat(ingest): SageMaker jobs and models
• #2847 @hsheth2 feat(docs): throttle and retry requests in doc generation
• #2842 @kevinhu fix(ingest): check for dbt materialization before proceeding
• #2840 @dexter-mh-lee fix(analytics): Fix SSL issue with analytics on frontend
• #2845 @hsheth2 feat(ingest): prettify stack traces in CLI
• #2851 @frsann fix(ingest): Fix glob pattern and handle possible recursion in lookml
• #2846 @hsheth2 fix(docs): fix some broken links in the docs
• #2843 @hsheth2 fix(ingest): avoid setting timestamps unless source system provides it
• #2841 @topwebtek7 feat(react): fix apollo client cache issues in entities profile pages
• #2832 @datascienceChris feat (helm datahub-gms): add ingress template to datahub-gms helm chart
• #2833 @kevinhu fix(ingest): do not fail dbt ingestion when encountering missing nodes
• #2837 @hsheth2 docs(ingest): clarify that the Kafka options are pass-through
• #2836 @hsheth2 fix(ingest): various BigQuery source fixes
• #2835 @hsheth2 fix(ingest): mask password in info-level logs
• #2834 @hsheth2 docs(ingest): update links to Kafka docs
• #2829 @shirshanka docs: update H2 2021 roadmap
• #2824 @gabe-lyons fix(mae-consumer): support standalone mae consumer without neo4j
• #2825 @hsheth2 fix(cli): change docker nuke to also remove stopped containers
• #2828 @hsheth2 feat(ci): separate metadata-ingestion into a separate workflow
• #2827 @hsheth2 fix(ingest): convert superset timestamps to micros
• #2826 @gabe-lyons docs(elastic-for-graph): Add migrating from neo4j to elastic instructions
• #2822 @hsheth2 docs(docker): fix check command reference
• #2823 @jjoyce0510 fix(docs): update OIDC docs
• #2817 @hsheth2 docs(ingest): add extra info for Redshift behind a proxy
• #2816 @hsheth2 fix(ingest): view handling resilience for redshift
• #2818 @jjoyce0510 feat(datahub-frontend): Adding basic file-based authentication to datahub-frontend
• #2814 @dexter-mh-lee chore(helm): upgrade version to v0.8.5

DataHub v0.8.5

Release Highlights

• Various stability fixes for v0.8.4
• Address docker image vulnerabilities
• New integrations: AWS SageMaker
• Support for restoring indexes with how-to
• Ingestion improvements: mongodb, looker, hive, snowflake

Changelog

• #2813 @dexter-mh-lee fix(datahub-upgrade): fix vulnerabilities
• #2804 @hsheth2 feat(ingest): basic support for complex hive types
• #2810 @dexter-mh-lee fix(k8s): upgrade helm version
• #2812 @hsheth2 feat(ingest): refactor mce comparison and add pytest update golden files option
• #2809 @hsheth2 docs(website): hide outdated FAQs page
• #2779 @dexter-mh-lee feat(backup): Add restore indices and restore backup tasks
• #2793 @hsheth2 feat(ingest): support ingesting from multiple snowflake dbs
• #2811 @jjoyce0510 fix(quickstart): Fixing manual mysql quickstart
• #2786 @hsheth2 refactor(ingest): extract sqlalchemy uri generation logic
• #2803 @dexter-mh-lee fix(usage-stats): Add usage stats factory to mae processor config
• #2800 @hsheth2 fix(ingest): better warnings and error handling for rest sink
• #2758 @kevinhu feat(ingest): SageMaker feature store ingestion
• #2770 @remisalmon feat(ingest): Improve lookml sql derived tables detection, add cascading derived tables to lineage
• #2781 @dexter-mh-lee fix(search): Filter out "removed" entities from autocomplete and analytics
• #2799 @jjoyce0510 fix(datahub-upgrade): add runtime dependency on logbackClassic
• #2801 @hsheth2 fix(ingest): quote table names in hive
• #2796 @hsheth2 fix(ingest): handle case when view definition handler is not implemented
• #2798 @dexter-mh-lee fix(usage-stats): add indices target for usage stats query
• #2797 @hsheth2 refactor(ingest): remove deprecated methods and warn on deprecated import
• #2776 @hsheth2 docs(website): add releases page
• #2794 @jjoyce0510 Update quickstart to include system prune
• #2783 @kevinhu fix(ingest): use correct platform for MongoDB ingestion
• #2785 @jjoyce0510 fix(frontend): making nested SSL configs optional
• #2792 @dexter-mh-lee feat(k8s): Upgrade to v0.8.4
• #2780 @kevinhu fix(docs): links to Feast entities
• #2782 @hsheth2 refactor(ingest): use common get_sys_time method
• #2784 @dexter-mh-lee fix(docker): Upgrade to 4.1.44 netty-all library
• #2778 @kevinhu feat(ingest): add non-random sampling for mongo
• #2774 @hsheth2 docs: upgrade docusaurus, minor ingestion updates

DataHub v0.8.4

Release Highlights

• Dataset Popularity, Recent Queries powered by Usage logs (support for Snowflake, BigQuery)
• Markdown descriptions and editing
• New Integrations : Glue Jobs, Feast
• Versioned API for metadata GETs
• No neo4j requirement, Elastic for Graph
• Docker image hardening
• Improved logging
• GCP Deployment Guide

Changelog

• #2773 @jjoyce0510 feat(logs): add thresholding, misc cleanup
• #2771 @topwebtek7 fix(react): update platform text in dataset profile header
• #2772 @dexter-mh-lee fix(nocode): removing service PDL
• #2761 @jjoyce0510 feat(logs): improve logging in GMS and datahub-frontend
• #2769 @topwebtek7 fix(react): fix graphql apollo cache update issue cause of usagestats
• #2768 @dexter-mh-lee feat(k8s): add GCP deploy recipe
• #2767 @hsheth2 fix(react): update the query frequency text label
• #2766 @hsheth2 docs(ingest): move usage stats docs into the "sources" section
• #2728 @jjoyce0510 fix(datahub-upgrade): removing the CleanupStep from datahub upgrade
• #2763 @dexter-mh-lee fix(docker): Fix dependency vulnerability
• #2765 @hsheth2 fix(react): move percent sign after number and update meta tag
• #2695 @thomasplarsson fix(frontend): auth session ttl specified in hours instead of days. M…
• #2764 @hsheth2 docs: update for Jun townhall
• #2762 @hsheth2 feat: usage stats (part 2)
• #2750 @hsheth2 feat: usage stats (part 1)
• #2759 @topwebtek7 fix(react): reverse result of topological sort, autofocus in add tag modal
• #2753 @gabe-lyons feat(elastic-as-graph): defaulting to elastic in quickstart
• #2760 @hsheth2 fix(docker): use head tag for datahub-ingestion
• #2754 @gabe-lyons Revert "fix(gms): add rest.li validation in gms (#2745)"
• #2751 @dexter-mh-lee fix(browse): sort by doc count descending
• #2752 @gabe-lyons fix(elastic-as-graph): adding elasticsearch setup back in
• #2739 @jjoyce0510 fix(gms): make get return entity type
• #2749 @remisalmon feat(ingest): add option to specify source platform database in lookml ingestion
• #2729 @kevinhu feat(ingest): ingest last-modified from dbt sources.json
• #2746 @dexter-mh-lee fix(docker): modernize docker images and fix vulnerabilities
• #2740 @jjoyce0510 fix(datahub-upgrade): add support for postgres migration
• #2745 @jjoyce0510 fix(gms): add rest.li validation in gms
• #2748 @jjoyce0510 feat(quickstart): remove orphaned docker containers on quickstart through cli
• #2744 @jjoyce0510 feat(docker): reduce quickstart footprint
• #2747 @kevinhu fix(ci): increase wait-for-it timeout to fix flaky feast test
• #2743 @kevinhu feat(ingest): print docker logs on timeout
• #2726 @gabe-lyons feat(graph): support using elasticsearch as graph backend.
• #2723 @gabe-lyons fix(gms): fixes for version aspect fetching
• #2741 @hsheth2 fix(ingest): update lookml test
• #2742 @kevinhu fix(ingest): fix lookml platform URN
• #2687 @kevinhu feat(ingest): add support for Glue ETL jobs
• #2716 @kevinhu fix(ingest): types for dbt
• #2722 @kevinhu fix(ci): increase Feast docker setup timeout
• #2737 @remisalmon fix(looker): fix invalid URN syntax error
• #2730 @dexter-mh-lee fix(docker): update default tags to head
• #2727 @gabe-lyons fix(docs): update extending-the-metadata-model.md
• #2721 @gabe-lyons fixing docs sidebar
• #2719 @dexter-mh-lee Update version for helm

DataHub v0.8.3

Release Notes

Bug fix release that fixes editable descriptions bug from previous release.

Previous version release notes: https://github.com/linkedin/datahub/releases/tag/v0.8.2

Changelog

#2718 @topwebtek7 fix(react): update schema description edit behavior

DataHub v0.8.2

Release Notes

Bug fix release that fixes installation, upgrade and usability issues with v0.8.1 specifically around product analytics.
Read the release notes for v0.8.1 here.
Full list of improvements follow.

Changelog

• #2711 @jjoyce0510 fix(noCode): Improving efficiency of EntityService "listLatestAspects" API
• #2715 @hsheth2 feat(quickstart): use versioned docker images when on a release tag
• #2710 @gabe-lyons fix(editable descriptions): adding indexing for editable descriptions
• #2705 @vijayan-nallasami-curve added node_type_pattern in dbt yaml file
• #2702 @vijayan-nallasami-curve Removed key not in catalog filter from dbt and added node filter using AllowDenyPattern
• #2712 @hsheth2 feat(ingest): expose additional types to Python via codegen
• #2674 @kuntalkumarbasu feat: Adding annotation option to frontend service
• #2699 @hsheth2 test(ingest): simplify docker cleanup commands
• #2708 @hsheth2 fix(ingest): use looker data platform
• #2709 @kevinhu feat(ingest): Add test case and docs for SQL view ingestion
• #2707 @topwebtek7 feat(entities): add markdown description update/viewer feature in dataset, datajob, dataflow, chart and dashboard, update ui/ux
• #2677 @gabe-lyons feat(aspects): support fetching of versioned aspects
• #2691 @kwark Add support for different oidc client authentication methods
• #2701 @JeffSkinner feat(gms): support basic auth header when connecting to elasticSearch
• #2697 @dexter-mh-lee fix(NoCode): Fix product analytics
• #2690 @jjoyce0510 feat(gms): Merge MAE, MCE consumers into GMS
• #2689 @jjoyce0510 feat(datahub cli): DataHub CLI Quickstart
• #2672 @kevinhu feat(docs): Docs for S3 ingestion with AWS Glue
• #2686 @remisalmon Only work with dbt catalog data if load_catalog is False
• #2688 @gabe-lyons removing whitespace from service aspect
• #2663 @vlavorini feat(sql_views): added views as datasets for SQLAlchemy DBs
• #2676 @hsheth2 feat(ingest): support Oracle service names
• #2684 @hsheth2 fix(ingest): upgrade acryl-pyhive to use sasl3 instead of sasl
• #2632 @RickardCardell Fix 2592 neo4j connection options
• #2670 @hsheth2 fix(ingest): pin to new mypy version
• #2637 @kevinhu feat(ingest): headers for codegen Python scripts
• #2675 @thomasplarsson fix(analytics): Support sasl authentication to kafka
• #2671 @zack3241 feat(k8s): Add imagePullSecrets to all K8's jobs
• #2667 @zack3241 Add get_identifier to hive source in metadata ingestion
• #2669 @gabe-lyons fix(no-code): Adding Chart input relationship annotations
• #2605 @kevinhu feat(ingest): Feast ingestion integration
• #2666 @kevinhu fix(ingest): fix MyPy stubs
• #2661 @hsheth2 build(docker): test docker builds in pull request CI
• #2664 @dexter-mh-lee docs(nocode): fix typo in autocomplete annotation
• #2662 @gabe-lyons docs(nocode): adding documentation to handle backwards incompatibility issues
• #2660 @hsheth2 fix(ingest): exclude mssql-odbc from "all" extra
• #2655 @dexter-mh-lee fix(NoCode): Update snapshot json to latest
• #2657 @hsheth2 fix(ingest): support mssql encryption via ODBC
• #2658 @hsheth2 fix(docker): use debug tag in local dev images
• #2659 @hsheth2 revert: "fix(docker): pin containers to golden hash for release (#2654)"
• #2656 @gabe-lyons docs(nocode): Adding documentation for no-migration upgrade option
• #2643 @dexter-mh-lee feat(nocode): Add datahub-upgrade job to helm chart and set version to v0.8.1

DataHub v0.8.1

Release Notes

• Bug fix release that fixes installation and upgrade issues with v0.8.0.
• Read the release notes for v0.8.0 here.

Changelog

• #2654 @hsheth2 fix(docker): pin containers to golden hash for release
• #2653 @jjoyce0510 feat(datahub-upgrade): improve no code upgrade logging
• #2651 @hsheth2 docs(nocode): fix links to datahub-upgrade
• #2646 @jjoyce0510 fix(gms): Return empty snapshots when one does not exist
• #2652 @jjoyce0510 fix(datahub-upgrade): fixing mis-spelled schema registry url
• #2650 @hsheth2 fix: use head tag by default in quickstart
• #2649 @dexter-mh-lee docs(nocode): Update migration docs with workaround for stuck upgrade
• #2648 @dexter-mh-lee fix(nocode): Fix docker image tag for run_upgrade script
• #2647 @dexter-mh-lee fix(analytics): fix index names
• #2645 @hsheth2 fix(build): don't purge ingestion codegen files on gradle clean
• #2640 @jjoyce0510 fix(datahub-upgrade): remove debug flag
• #2641 @jjoyce0510 feat(datahub-upgrade): Use the "head" image tag.
• #2642 @gabe-lyons docs(no-code): removing unused snapshot annotations

DataHub v0.8.0

Notable Highlights

• Product Analytics : Understand how your users are interacting with DataHub
• Product Improvements: Auto-complete across types, Task list view under Pipelines
• Features: Business Glossary (incubating)
• Integration improvements
  • Looker, dbt, Hive, Redshift, Glue, MongoDB
  • Kafka Connect (incubating)

and finally,

NoCodeMetadata

This release introduces a major refactor that permits extension of DataHub’s metadata model without writing any imperative code.

Highlights:

• Removed strongly-typed, entity-specific DAOs. Added more generic services.
• Introduced Elastic settings & mappings generation, dynamic index registration & evolution
• Decoupled persistence layer from Pegasus + Java by removing fully-qualified class names (aspects, relationships)
• Introduced declarative, annotation-based mechanisms for defining indexed fields, foreign key fields, entities & aspects
• In-place upgrade CLI to aid in adopting this upgrade (datahub-upgrade)

For more information, see

The PR: #2629
Technical Overview
The DataHub Metadata Model
Extending the Metadata Model
No Code Upgrade Guide

ChangeLog

• #2629 @jjoyce0510 feat: No Code Metadata Modeling
• #2617 @shirshanka docs: update roadmap with accomplished items
• #2635 @hsheth2 fix(ingest): improve redshift ingestion performance
• #2599 @shubham49 feat(react): replace user urn with username
• #2623 @gabe-lyons fix(react): url encoding urns and tag profile fix
• #2634 @hsheth2 fix(ingest): include urn as key for kafka emitter
• #2636 @dexter-mh-lee fix(ci): update trigger to always generate docker images
• #2622 @RickardCardell feat(react): custom properties are now sortable by name in the UI
• #2626 @thomasplarsson fix(ingestion): improve robustness of glue ingestion source
• #2619 @topwebtek7 fix(react): update ispartofbuilderfromdataflow, update ui in datajob header
• #2620 @jjoyce0510 feat(analytics): support configuration of Kafka SSL
• #2618 @topwebtek7 feat(react): eliminate noises in react build, test and cleanup, get rid of warnings
• #2616 @dexter-mh-lee docs: added AWS deployment guide
• #2615 @gabe-lyons fix(react): fixing tags autocomplete bug
• #2590 @saxo-lalrishav feat(react): business glossary and user - tab based profile page
• #2612 @hsheth2 docs: update homepage text
• #2614 @hsheth2 fix(ingest): fail gracefully when lookml used on old python versions
• #2603 @topwebtek7 feat(react): update collectionname in datajob, update tabs ui/ux
• #2604 @topwebtek7 feat(graphql): redesign autocomplete to search for all entity types, show suggestions grouped by entities
• #2606 @hsheth2 feat(ingest): populate inputDatajobs field in airflow integration
• #2602 @topwebtek7 feat(react): add parent flow link on datajob page
• #2596 @remisalmon fix(ingest): fix lineage after dbt metadata ingestion when tables name and identifier differ
• #2600 @topwebtek7 feat(react): add topological sort feature, update graphql, add tests
• #2607 @hsheth2 feat(ingest): update bigquery demo data
• #2609 @hsheth2 fix(docs): various fixes and additions
• #2601 @frsann feat(ingestion): Fix looker test
• #2577 @shubham49 feat(react): add glossary term to dataset preview
• #2585 @afranzi fix(ingest): incorrect implementation of the allow pattern in looker dashboards
• #2591 @martha feat(react): add optional subtitle to home page
• #2598 @kevinhu fix(ingest): default values for env
• #2575 @hsheth2 docs(ingest): add a guide for writing sources
• #2594 @topwebtek7 feat(react): add nativeDataType with tooltip over icon in schema
• #2595 @havramar docs: Add Plum Research to POC adoption section in README.md
• #2589 @martha feat(react): prevent logo distortion
• #2586 @gabe-lyons fix(react): fix tag autocomplete after creating a new tag
• #2583 @bboylen feat(react): Add label to edited dataset descriptions
• #2579 @topwebtek7 feat(dataflow): update dataflow to have datajobs in new tab
• #2584 @john-bodley fix(docs): Fix Superset typo in README
• #2574 @zack3241 fix(helm charts): remove connection tests from helm charts
• #2582 @hsheth2 build(ingest): show diff upon lint failures
• #2516 @taufiqibrahim feat(ingest): kafka connect metadata ingestion
• #2580 @hsheth2 feat(ingest): add dataset tag transformer
• #2573 @hsheth2 test(ingest): use different mysql test port
• #2549 @shubham49 feat(react): link glossary term to dataset page
• #2572 @hsheth2 test(ingest): ensure transformer registry works for aliases
• #2571 @hsheth2 fix(ingest): better active directory LDAP support
• #2483 @luck02 fix(dbt): set target platform and load schema
• #2563 @afranzi feat(ingest): add AWS IAM Roles Support to the Glue Source
• #2566 @saxo-lalrishav fix(react): Update raw schema view to support non json schemas
• #2570 @saxo-lalrishav fix(react): Removing a user having multiple role from owner tab also remove the other roles associated to that user
• #2562 @sunkickr docs: Add Sphinx Docstrings to Airflow Modules
• #2560 @hsheth2 fix(cli): prevent click from suppressing errors
• #2559 @hsheth2 docs: include license in the readme
• #2561 @hsheth2 fix(ingest): check mypy types for test helpers
• #2558 @shirshanka docs: town-hall updates and some badges
• #2557 @hsheth2 feat(ingest): add options for Airflow lineage backend
• #2467 @pedro93 feat(k8s): generalizes CronJob metadata ingestion resource for custom logic
• #2546 @kevinhu feat(ingest): MongoDB schema inference
• #2556 @gabe-lyons fix(search): have search bar ignore blank searches
• #2553 @gabe-lyons fix(owner): fixing ownership routing
• #2555 @gabe-lyons feat(business glossary): hiding business glossary until all features completed
• #2493 @frsann feat(ingest): Looker view and dashboard ingestion
• #2538 @saxo-lalrishav feat(business glossary): search, browse and entity page for business glossary terms
• #2543 @hsheth2 fix(ingest): register custom Hive types
• #2544 @hsheth2 docs(ingest): improve kafka schema registry config docs
• #2545 @G-nther fix(analytics): use seperate env variable for tracking topic in MAE-Consumer
• #2547 @hsheth2 ci(docker): disable GitHub Docker registry
• #2521 @hsheth2 refactor(ingest): move Airflow into datahub_provider module
• #2539 @dexter-mh-lee fix(analytics): add support for AWS ES
• #2540 @afranzi feat(ingest): define Redshift as a Postgres Source
• #2541 @jjoyce0510 fix(react): disable analytics link display
• #2542 @topwebtek7 fix(react): fix type issue with adding new in ownership
• #2531 @hsheth2 build(ingest): use gradle in commands + docs
• #2536 @hsheth2 fix(ingest): remove mce.json file from root
• #2535 @gabe-lyons fix(react): fixing import issue
• #2534 @kevinhu docs: autoplay and navigation for source logos carousel
• #2519 @topwebtek7 feat(usergroup): implement corpgroup in graphql, refactor avatars and ownership in react
• #2532 @hsheth2 feat(ingest): add a transformer for adding ownership
• #2485 @shubham49 feat(graphql): add graphql types for business glossary
• #2533 @dexter-mh-lee fix(k8s): Fix helm charts for supporting analytics
• #2499 @jjoyce0510 feat(Product Analytics): Introducing In-App Analytics Beta
• #2529 @hsheth2 docs: enable better syntax highlighting
• #2528 @kevinhu docs: Use carousel layout for ingestion source logos
• #2530 @jjoyce0510 fix(model): removing reference to go link in SchemaFieldPath model
• #2515 @hsheth2 docs: update docusaurus
• #2527 @dexter-mh-lee fix(k8s): change defaultMode for certs volume
• #2503 @hsheth2 feat(ingest): check in generated schema files
• #2526 @nickwu241 fix(k8s): fix kafka-setup-job.yml datahub-certs-dir mountPath
• #2512 @dexter-mh-lee fix(k8s): comment out minikube specific settings
• #2522 @hsheth2 fix(ingest): generate Airflow tags correctly
• #2523 @topwebtek7 feat(react): set fixed height for dataset preview
• #2525 @kevinhu docs: Add ingestion source logos grid
• #2524 @hsheth2 fix(ingest): add support for custom postgres types
• #2228 @shakti-garg feat(business_glossary): add new entity business term and its relationship with dataset and its fields
• #2520 @hsheth2 fix(build): only check for src/ and tests/ directories for lint checks
• #2514 @hsheth2 docs: update Wolt logo
• #2513 @hsheth2 build(ingest): include package data in sdist
• #2510 @hsheth2 build(ingest): add metadata-ingestion to gradle build
• #2509 @hsheth2 docs: improve airflow explanations and examples
• #2508 @hsheth2 fix(ingest): remove double edges from Airflow lineage backend
• #2505 @vlavorini docs: fixed MCE file recipe example
• #2500 @dexter-mh-lee docs(k8s): Update readme with helm prerequisite
• #2501 @gabe-lyons feat(lineage): removing dataset<>dataset edge in job...

DataHub v0.7.1

Notable Highlights

• Lineage Visualization
• Pipelines and Tasks, Flows and Jobs
• Airflow Lineage
• Editable Field Descriptions
• Nested Schema Viz
• Search Improvements
• datahub CLI
• Official PyPi packages
• Production-quality Helm scripts
• New Integrations
  • Officially-supported Sources: Airflow, AWS Glue, dbt, Druid, Superset, MongoDB, Oracle

Changelog

• #2440 @dexter-mh-lee feat(k8s): Move helm charts out of contrib
• #2397 @gabe-lyons feat(lineage): implement support for datasets, charts and dashboards downstream lineage fetching in a generic way
• #2434 @adriaanslechten feat(ingest) LDAP groups ingestion
• #2438 @hsheth2 fix(ingest): use entrypoints lib instead of pkg_resources
• #2425 @gabe-lyons feat(ingest): adding superset ingestion source
• #2433 @topwebtek7 fix(react): fix lineage sidebar buttons
• #2436 @hsheth2 fix(ingest): support custom snowflake types
• #2419 @topwebtek7 feat(react): add dataJob, dataFlow entity pages, refactor with fragments
• #2418 @frsann Fix(search): fix datajob and dataflow search mappings
• #2429 @hsheth2 fix(ingest): fix chart type enum serialization and add tests for rest emitter
• #2431 @shirshanka docs: Update agenda for Apr 23 townhall
• #2427 @hsheth2 fix(ingest): ensure upstreams in airflow lineage emission are entities
• #2426 @hsheth2 fix(ingest): include database info for snowflake
• #2424 @hsheth2 feat: add s3 data platform and logo
• #2423 @topwebtek7 feat(react): schema visualization add support for nested structs
• #2422 @topwebtek7 fix(react): lineage sidebar buttons should refer to the selected entity
• #2421 @dexter-mh-lee fix(kafka-setup): Fix start script for kafka setup
• #2417 @topwebtek7 feat(react): update dataset entity default svg icon
• #2411 @thomasplarsson feature(ingestion): Adding the concept of transformers
• #2415 @dexter-mh-lee fix(k8s): Add credentials to kafka-setup job and clean up
• #2412 @hsheth2 feat(ingest): add Kafka-based emitter example
• #2413 @gabe-lyons fix(lineage): allow lineage viz to handle circular dependencies
• #2414 @dexter-mh-lee fix(kafka-setup): Add the correct context to the git workflow for pushing kafka-setup image
• #2403 @hsheth2 fix(ingest): bump avro-gen3
• #2406 @topwebtek7 feat(react): use default entity icon if lineageentity has no icon
• #2408 @hsheth2 fix(ingest): properly handle fieldDiscriminator with restli
• #2409 @hsheth2 fix(ingest): add sqlalchemy extra
• #2398 @G-nther feat(kafka-setup): add option for SSL and topic partition config via environment
• #2404 @dexter-mh-lee feat(k8s): add extraEnvs to setup jobs
• #2407 @topwebtek7 feat(react): add footer buttons in lineage sidebar
• #2405 @thomasplarsson feature(ingestion): Make origin/fabric_type configurable
• #2384 @topwebtek7 feat(react): add padding between tags and description on datasets profile page
• #2396 @gabe-lyons feat(sample): adding sample mces for dataflows and datajobs
• #2400 @hsheth2 fix(ingest): streamline codegen init methods
• #2382 @topwebtek7 feat(react): update schema table to have fixed description column, set line break with max description width
• #2402 @dexter-mh-lee fix: Fix env variable setup for kafka, mysql-setup docker containers
• #2401 @hsheth2 fix(ingest): add db name to postgres URNs
• #2393 @hsheth2 fix(ingest): enable mypy disallow_incomplete_defs and disallow_untyped_decorators
• #2395 @gabe-lyons fix(react): fix access to pictureLink in charts and dashboards
• #2399 @gabe-lyons fix(tags): check description existence on tags
• #2383 @topwebtek7 feat(react): fix long descriptions overflow issue in lineage side panel
• #2392 @hsheth2 refactor(ingest): update test harness to use a compose file per test
• #2391 @topwebtek7 feat(react): fix browse link of last breadcrumb linked to unknown page
• #2385 @dexter-mh-lee feat(mysql-setup): Add the ability to specify database name for mysql-setup
• #2389 @hsheth2 feat(ingest): add generic sqlalchemy source
• #2390 @dexter-mh-lee feat(k8s): Add ability to add service accounts to setup jobs
• #2387 @dexter-mh-lee fix(kafka-topic-convention): Fix DAOs that do not refer to TopicConvention
• #2386 @dexter-mh-lee feat(index): Add index naming convention for elasticsearch
• #2388 @hsheth2 fix(ingest): report correct version status in dev mode
• #2368 @hsheth2 feat(ingest): add Airflow lineage backend
• #2380 @OddCN fix(docs): fix config example for file sink
• #2362 @dexter-mh-lee feat(k8s): Update pods with correct probes and remove unnecessary dependencies
• #2372 @thomasplarsson fix(ingestion): dont crash on non-RecordSchema topics
• #2360 @hsheth2 docs(ingestion): remove outdated data-source-onboarding.md docs
• #2376 @topwebtek7 feat(react): hide Owned By label in card if no owners
• #2373 @shubham49 fix(react): ownership rendering
• #2377 @topwebtek7 feat(react): add null state indicator in user profile when no entities
• #2379 @topwebtek7 feat(react): update avatar to use initial if no image, refactor all avatars with custom one
• #2369 @gabe-lyons feat(lineage): support arbitrary entity types in lineage viz
• #2364 @thomasplarsson fix(ingestion): Support mapping from avro "boolean" and "map" types t…
• #2343 @thomasplarsson fix(ingestion): properly detect optional fields in avro schemas
• #2370 @topwebtek7 feat(react): add empty state UI for browse when no entities
• #2242 @frsann feat(datajob): Datajob graphql query
• #2367 @topwebtek7 feat(react): add dropdown menu links, menu styling, removed warnings
• #2365 @frsann chore(dependabot): Update pyyaml version
• #2366 @topwebtek7 feat(react): add icons on entities, updated styling in LineageViz
• #2351 @hsheth2 fix(ingest): add test for avro serialization and deserialization
• #2361 @hsheth2 feat(cli): Add support for checking docker memory usage
• #2358 @topwebtek7 feat(react): original description shows in edit modal even when the description has been updated
• #2357 @gabe-lyons feat(react): improving error logging on dataset entity
• #2356 @dexter-mh-lee fix(elasticsearch): Fix inconsistencies between documents and elasticsearch mappings
• #2359 @hsheth2 fix(ingest): support python3 -m datahub
• #2353 @hsheth2 chore(ingest): remove unused plugin_requirements.txt file
• #2352 @hsheth2 fix(ingest): bump pybigquery version
• #2350 @hsheth2 fix(ingest): support datahub --version
• #2349 @gabe-lyons feat(lineage): improve lineage re-focus experience
• #2341 @frsann feat(tags): Add tag graph builder
• #2348 @jjoyce0510 fix(Ember App): Allow ember build (disabled by default)
• #2345 @hsheth2 fix(cli): add --verbose flag for datahub check plugins
• #2346 @gabe-lyons fix(lineage): add upstream arrows back in
• #2347 @hsheth2 feat(ingest): add Oracle db support
• #2336 @topwebtek7 feat(react): add description edit behavior along with modal
• #2340 @gabe-lyons feat(lineage): adding ghost edges indicating hidden dependencies
• #2331 @hsheth2 feat(ingest): start airflow integration + metadata builders
• #2339 @hsheth2 fix(ingest): add support for database and table patterns to glue source
• #2338 @hsheth2 fix(docker): remove restart: always from docker-compose for consistency
• #2335 @gabe-lyons feat(lineage): adding directionality to lineage edges to make the visualization more clear
• #2337 @gabe-lyons fix(lineage): fixing lineage layout bugs
• #2319 @amonkhouse feat(ingest): adding support for AWS Glue
• #2312 @shakti-garg feat(es-setup): add logic in elasticsearch setup to compare-and-update index if already exists
• #2333 @gabe-lyons feat(lineage): expandable lineage visualization for dataset <> dataset lineage
• #2332 @hsheth2 docs: add wolt logo to frontpage
• #2315 @grantatspothero feat(ingest): adds experimental support for ingesting Looker metadata
• #2330 @luck02 fix(test): dbt-manifest files
• #2329 @topwebtek7 feat(react): moving filter panel from modal to drawer
• #2328 @hsheth2 build: remove deprecated ember app from build
• #2327 @hsheth2 feat(ingest): verify dynamic registry types at runtime
• #2316 @joemirizio feat(ingest): dynamically register plugins
• #2325 @hsheth2 fix(ingest): remove outdated metadata-ingestion scripts
• #2313 @shakti-garg fix(k8s): make es-setup job parameters more contextual
• #2322 @gabe-lyons docs(theme): making yarn start instructions more explicit
• #2317 @hsheth2 doc: update slack links to https
• #2324 @frsann fix(datajob): Fix URI templates for datajob and dataflow
• #2320 @frsann fix(tags): Support creating tags with MCE
• #2323 @arunvasudevan fix(docs): Update metadata-serving.md
• #2318 @dexter-mh-lee fix(docker): Fix issue in gms start.sh
• #2321 @shirshanka docs: Update next townhall details, fixup links and misc docs
• #2251 @bernardino feat(Kubernetes): Add JMX exporter containers to all DataHub components
• #2308 @dexter-mh-lee fix(search): Fix styling for column match snippet
• #2302 @shakti-garg feat(k8s): Add k8s hook in datahub helm chart for setting up elasticsearch
• #2298 @dexter-mh-lee feat(docker): Add dockerfile for initializing an existing mysql server
• #2297 @shakti-garg feat(kafka-config): add variable KAFKA_CONSUMER_GROUP_ID to ove...

DataHub v0.7.0

Notable Highlights

• New React Application re-written from the ground up
• Support for GraphQL
• New Metadata Ingestion Framework (Python)
  • Officially-supported Sources: Kafka, MySQL, SQL Server, Hive, Postgres, Snowflake, BigQuery, AWS Athena, Druid, LDAP
• New Homepage and Hosted Docs redesign at datahubproject.io
• Product Features: SSO (OIDC), Tags, Themes, Dashboards
• Metadata Backend Implementations: MLModel ecosystem, DataFlow ecosystem
• Move to Elasticsearch 7. Migration guide from 5.x here

Changelog

• #2263 @jplaisted feat(search) BREAKING Support ElasticSearch 7, drop ES5
• #2260 @gabe-lyons fix(tags): fixing margins on tags for long descriptions
• #2259 @hsheth2 docs: update roadmap progress
• #2258 @dexter-mh-lee refactor(demo): Add empty global tags to BigQuery demo data
• #2255 @jjoyce0510 feat(react): Adding shadow and deeper linear gradient
• #2254 @gabe-lyons feat(tags): improving elastic search templates for tags
• #2253 @gabe-lyons fix(tags): fix ownership on tag create
• #2256 @hsheth2 fix: update slack links
• #2248 @gabe-lyons feat(tags): editing tags from react client on datasets, schemas, charts & dashboards
• #2252 @jjoyce0510 refactor(react): React as the default UI
• #2246 @hsheth2 feat(ingest): various minor fixes
• #2245 @jjoyce0510 feat(react): Adding big query logo
• #2249 @gabe-lyons fix(react): enabling charts and dashboards to be supported by theme config
• #2235 @pedro93 feat(ingest): Add support for druid
• #2244 @gabe-lyons feat(react): moving schema tab to be default
• #2243 @shirshanka docs: adding mar-19 townhall agenda
• #2240 @dexter-mh-lee feat(tags): Enable search for datasets by tags
• #2236 @pedro93 feat(k8s): Add metadata-ingestion as a Helm component
• #2241 @shirshanka docs: Improving architecture docs
• #2239 @hsheth2 feat(docs): use gradle for building docs
• #2232 @hsheth2 fix(ingest): various avro codegen fixes
• #2237 @gabe-lyons fix(dataflow): fixing browse dao access
• #2166 @arunvasudevan feat: MLmodel Graphql Query
• #2197 @frsann feat(datajob): Backend implementation
• #2233 @jjoyce0510 refactor(react): All entity search UI + misc improvements
• #2234 @jjoyce0510 docs(react): Oidc React Doc Updates
• #2231 @dexter-mh-lee fix(docker): start issue when there are multiple kafka brokers in bootstrap config
• #2227 @jjoyce0510 refactor(React): Misc UI improvements
• #2230 @hsheth2 fix(ingest): pin version of avro-gen3
• #2226 @hsheth2 fix(ingest): use python extras in docker image
• #2224 @hsheth2 feat(ingest): use plugin system based on Python extras
• #2190 @jjoyce0510 feat(react): SSO support simple OIDC authentication
• #2223 @dexter-mh-lee Added images to es/kafka-setup
• #2222 @dexter-mh-lee fix(ci): rename file to match git workflow needs
• #2220 @dexter-mh-lee fix(ci): remove paths_ignore from workflow files
• #2219 @thomasplarsson refactor(ingest): improve athena source api and documentation
• #2221 @gabe-lyons fix(ci): setting CI to false for builds
• #2218 @gabe-lyons feat(react): hiding raw schema button when no raw schema exists
• #2216 @dexter-mh-lee fix(es-setup): Add git workflows to upload docker for elasticsearch and kafka setup
• #2213 @thomasplarsson feat(ingest): add aws athena ingestion source
• #2217 @gabe-lyons fix(ci): fail CI on react build errors
• #2215 @gabe-lyons fix(react): fix theming test in react and simplifying api
• #2209 @thomasplarsson feat(ingest): add option for optimized skipping of schemas
• #2212 @hsheth2 fix(ingestion): nullable types and timestamp precision
• #2207 @hsheth2 feat(ingest): standalone metadata emitters
• #2205 @dexter-mh-lee fix(ci): Fix github package path
• #2204 @dexter-mh-lee feat(ci): Add SHA based tagging before pushing to docker registries
• #2203 @gabe-lyons feat(tag): adding search for tags in gms layer
• #2193 @gabe-lyons feat(react): adding ability to support theming of datahub, with two themes included
• #2201 @hsheth2 feat: add date and time types to SQL model
• #2202 @thomasplarsson feat(mae-consumer): enable mae-consumer to use ssl when communicating with elasticsearch
• #2199 @thomasplarsson fix(mae-consumer): mae-consumer needs sslcontext bean
• #2181 @shirshanka chore: renaming business_glossary rfc directory with pull request number
• #2182 @shirshanka chore: renaming graphql_frontend rfc directory with pull request number
• #2183 @shirshanka chore: renaming react-app rfc directory with pull request number
• #2196 @shirshanka docs(roadmap): update project roadmap
• #2195 @jjoyce0510 fix(graphql): Add "fixed" SchemaFieldDataType mapping
• #2194 @gabe-lyons feat(tags): Enriching sample data for tags
• #2191 @hsheth2 feat(docs): automatically populate sidebar with RFCs
• #2192 @jplaisted (feat) Simple python script to carry over ES indices from 5 to 7.
• #2173 @brendansun93 feat(React): Ownership component of user profile
• #2189 @thomasplarsson feat(gms): add elasticsearch SSL support
• #2112 @frsann feat(tags): RFC for tags
• #2187 @gabe-lyons fix(react): fixing test issues that arose from ill-timed merges
• #2164 @gabe-lyons feat(tags): adding support for read/write of tags in gms & read-only in react datahub-frontend.
• #2185 @jjoyce0510 feat(graphql): More forgiving for unknown data platforms during reads
• #2184 @jjoyce0510 test(React): Home page tests
• #2186 @hsheth2 fix(docs): fix broken links
• #2179 @gabe-lyons feat(react): adding raw schema view option for table schemas
• #2178 @hsheth2 feat(ingest): bigquery sample data
• #2176 @hsheth2 docs: point to hosted docs site
• #2177 @hsheth2 docs(ingest): clarify setuptools requirement
• #2175 @hsheth2 build(docs): only deploy docs on main repo
• #2174 @hsheth2 docs: hosted documentation website
• #2167 @jjoyce0510 feat(React): Impl browse UI for Dashboards and Charts
• #2168 @jjoyce0510 fix(React): Fix Browse Pagination Bug
• #2172 @hsheth2 fix(ingest): loosen Kafka broker validation
• #2165 @jjoyce0510 feat(DataPlatform Logos): Adding server driven logos
• #2171 @hsheth2 docs(ingest): clarify Kafka connection config
• #2169 @shirshanka doc(townhall): Add links for Feb 19, upcoming townhall on Mar 19
• #2161 @hsheth2 fix(ingest): bigquery source and dataset naming fixes
• #2163 @jjoyce0510 fix(graphql): Bubbling up exceptions logged in GraphQL resolvers
• #2159 @hsheth2 build(ingest): use multi-stage docker build for datahub-ingestion
• #2157 @hsheth2 feat(ingest): capture table descriptions
• #2158 @hsheth2 feat(ingest): switch quickstart to Python ingestion
• #2156 @pedro93 feat(ingest): support alternative authentication in sql ingestion
• #2152 @gabe-lyons fix(react): fixing format we propagate filters to graphql in
• #2154 @gabe-lyons feat(react): Redirecting /assets to index
• #2151 @hsheth2 build(docker): add large generated directories to dockerignore
• #2150 @hsheth2 ci(ingest): setup docker container for metadata ingestion
• #2145 @RickardCardell feat: neo4j Bolt TLS support (#2100)
• #2143 @dexter-mh-lee feat(dashboards): Add browse end point for charts and dashboards
• #2144 @RickardCardell feat: neo4j https support (#2101)
• #2147 @gabe-lyons docs(frontend): Update docs to clarify running local frontend w/ local react app
• #2148 @jjoyce0510 feat(gms): Add optional data platform display name
• #2149 @jplaisted Switch GMA dep from bintray to artifactory.
• #2146 @jjoyce0510 Fixing required audit stamps bug
• #2140 @jjoyce0510 feat(React): Search page UI improvements, 'all' entity search.
• #2133 @thomasplarsson feat(datahub-dao): enable services to access gms over https
• #2136 @hsheth2 feat(ingest): support Postgres PostGIS extensions
• #2139 @gabe-lyons docs(Ownership): making lack of support for ownergroups in frontend explicit in pdl
• #2137 @dexter-mh-lee refactor(docker-dev): set up elasticsearch using local mapping on docker-compose.dev
• #2135 @hsheth2 ci(ingest): run apt update
• #2134 @hsheth2 refactor(ingest): cleanup configuration models
• #2130 @jjoyce0510 feat(React UI): SearchPage and SearchResultsPage
• #2132 @jjoyce0510 Add URL to dashboard / chart page
• #2131 @gabe-lyons fix(React): Adding test coverage for search page & fixing filter select bug
• #2128 @jjoyce0510 fix(react): Fix authenticated user profile
• #2125 @hsheth2 fix(ingest): gracefully handle unknown types
• #2127 @jjoyce0510 feat: Introducing optional DataPlatform logo url
• #2124 @hsheth2 fix(ingest): update sample MCEs based on MLModel changes
• #2126 @jjoyce0510 fix(gms): fix getAllDataPlatforms bug
• #2123 @hsheth2 docs(ingest): add solutions for common install issues
• #2122 @hsheth2 feat(ingest): add support for LDAP ingestion
• #2120 @hsheth2 test(ingest): verify the output of mssql
• #2119 @jjoyce0510 feat(React): Adding basic chart + dashboard UI
• #2115 @brendansun93 feat(React): Avatar dropdown menu and logout function
• #2121 @hsheth2 feat(ingest): improve error reporting for pipelines
• #2117 @jjoyce0510 feat(GraphQL API): GQL implementation of Charts + Dashboards
• #2118 @...