The Drip | October 2022 Airbyte Product Updates

November 14, 2022

Summarize with AI:

Hey everyone, welcome to the October edition of the Drip where we take you downstream to cover highlights of our change-log, community and anything Airbyte related.

‍

Promotions & Shoutouts

Two connectors recently went into GA last month! Iterable, Files, Paypal Transactions, Recharge, and the Twilio connector saw this promotion. We also have some new connectors coming in from the community:

bit.io
GoCardless
Whiskey Hunter API
Gutendex API
Courier

We wanna give a shoutout to Adam Fletcher, Isaac from Urban Jungle, Jared Rhizor (OG Airbyter 😉), Dhroov Makwana, and Gabriel Samson for building out these connectors and showing the brand new low-code CDK some love.

Contributions from the community is what keeps us moving so if you have any connectors you would like to build, we strongly encourage it!

Security at top of mind

In our recent release, we added a default Nginx + HTTP Basic Auth proxy to Airbyte. What does this mean though?

If you haven't already done so, the next time you pull from the main branch, anytime you boot up Airbyte and head to localhost:8000, you'll be prompted to login prior to seeing the UI. This will help keep security at top of mind when working with Airbyte. Our docs and CLI have also been updated to reflect this change.

Head to this link to read more on this and feel free to ask us any questions that you may have.

Text4Shell

We have also upgrade the Apache Commons Text library from 1.9 to 1.10.0 due to CVE 2022-42889 (Text4Shell) being affected. This is done out of an abundance of caution, but we do not intend to back-release older versions because we aren't affected by the vulnerable behavior.

Low-code CDK

As mentioned above, we have been seeing a lot of love given to the new low-code CDK from everyone during Hacktoberfest. Your feedback has been greatly appreciated as we continue to build out this product to keep moving it in the right direction.

We hope to see this new scaffold continue to evolve and be the foundation for future projects we have down the line. Accessible data movement is the goal and we need your help in accomplishing that!

If you haven't tried out the low-code CDK, you can check out part 1 of our tutorial on Youtube as well as visit the docs for more information.

move(data) Conference

Last but not least, don’t forget to sign up for our very own move(data) conference (now Dec 7-8!) You’ll even get special insights and Octavia swag via our State of Data Engineering Survey!

Wrap Up

And that’s all we have for September’s edition of The Drip. Thanks for reading through. If you have any questions:

Please join our Slack community to talk to us on the Airbyte team as well as other fantastic folks in the community!
Also sign up for our Newsletter to keep up with the state of the art in Data Integration and the broader Data Engineering Ecosystem!

✨ New and improved features

New Sources and Promotions
Source Iterable.com to GA (#17668)!
Source File to GA (#17665)
🎉 New Destination: bit.io (#15821) - thanks Adam Fletcher!
🎉 New Source: GoCardless (#17792) - thanks Isaac from Urban Jungle!
🎉 New Source: Whisky Hunter API [low-code CDK] (#17918) - thanks Jared Rhizor, one of the OG Airbyters!
🎉 New Source: Gutendex API [low-code CDK] (#18075) - thanks Dhroov Makwana
🎉 New Source: Courier [low-code cdk] (#16847) - thanks Gabriel Samson! (another YC alum!)
New Features for Existing Connectors
Destination Redis: Add SSH support (#17951)
Destination Databricks: Support Azure storage (#15140) (#15329)
Source Amplitude: Add empty series validation (#17854)
Source Amazon Ads: filters for state on brand, product and display campaigns (#17475)
Source Github: improve 502 handling for comments stream (#17715)
Source Google Analytics v4: Added support for segments and filters (#16920)
Source Google Search Console: improve config validation - site_urls (#17751)
Source Iterable: Add permission check for stream (#17602)
Source Youtube analytics - added custom backoff logic (#17454)
Source MySql: Fix large table issue by fetch size (#17236)
Source MySQL : Expose serverTimezone debezium option via MySQL Source spec for CDC (#17815)
Source MongoDb: Increase performance of discover (#17614)
Source Shopify: Add metafield streams (#17962)
Source Zendesk Chat: engagements data fix infinity looping + gradlew format (#18121)
New Features in Airbyte Core
🪟 🔧 Add Datadog support to webapp, cleanup sentry init (#17821)
use basic auth username and password from .env (#18273)
Use Nginx + Basic Auth to secure OSS Airbyte (#17694)
🪟 🎉 Add /connector-builder page with embedded YAML editor (#17482)

🐛 Bug fixes

Source Bing Ads - Fix Campaigns stream misses Audience and Shopping (#17873)
Source Github: improve 502 handling for comments stream (#17715)
Source Google Sheets: fix nonetype exception when no spreadsheet id is found (#17766)
Source Mixpanel: solve cursor field none expected array (#17699)
Source MongoDb: Increase performance of discover (#17614)
Source MySql: Fix large table issue by fetch size (#17236)
Source Salesforce: make paging work, if cursor_field is not changed inside one page (#17615)
Source Shopify: fix 404 for configured streams, fix missing cursor error for old records (#17777)
Source Snapchat marketing: retry failed request for refreshing access token (#17596)
Source Orb: Fix bug to enrich multiple events with the same event_id (#17761)
Source Youtube analytics - added custom backoff logic (#17454)
Source Zendesk Chat: engagements data fix infinity looping (#17745)
Destination GCS: Fix error logs to log 'Gcs' rather than 'AWS' (#17901)