https://github.com/backstage/backstage/pull/8088#issuecomment-991183968, https://github.com/asyncapi/asyncapi-react/issues/498. Teams or Discord link, Make it possible for multiple Bazaar project to link to the same catalog entity. Here you can also search or add project to the Bazaar. For this, we created a file pattern that could be filled and versioned along with the microservice code, we called this file .checklist.yaml, and in it stored information about the team, squad, cost center, and description of which context that service was created. 7612e2856b: Clean up emptystate.svg image, removing wrong white artifact from the background, @backstage/plugin-permission-common@0.3.1, 9fbd3b90ae: fix: Register plugin to prioritise Component kind for entityRef, 451ef0aa07: Fix token pass-through for software templates using beta 3 version. Integrates with PagerDuty, Opsgenie, Jira, Google Docs, 30+ tools. As this data is specific and unique for each service and its context, we didnt want it to get lost in some documentation. With that, the documentation is quite easily searchable and accessible to anyone who needs it. We are excited to announce our Backstage integration on the Backstage Plugin Marketplace. Also, we have two main reasons to believe in the Backstage community: The right tool for the right problem, at the right time. 19f0f93504: Catch errors from a fact retriever and log them. Below we can see a simple representation of how the code looked in relation to the fields of the file: Due to the number of repositories and projects that we have in our CI system, adding a step in each pipeline was an unfeasible alternative for us. Thanks to Backstage core features, the spreadsheet for service ownership is no longer necessary. 14e980acee: ESLint upgraded to version 8 and all it's plugins updated to newest version. "@typescript-eslint/eslint-plugin": "^5.9.0". Understand how your cloud usage impacts our environment and what you can do about it via the Backstage developer portal. From the first challenge, we mentioned it was easy to create a frontend plugin for Backstage that would consume the addresses of our tools from an API we have internally, making all developers have access to our tools grouped in each context. The Backstage ecosystem and community surprised us. At QuintoAndar, we were challenged by two major problems: Below well describe the problems mentioned and how Backstage has helped us succeed. It handles multiple app configurations. The data needs to be stored and accessible so that we could analyze the services of each squad and tribe. This exciting announcement builds upon the collaboration we announced with Spotify last year to deliver developer portals to a number of clients, most notably with TELUS. Easy to install by leveraging the official Rootly + Backstage plugin: Import services created and managed in Backstage directly to Rootly, Create incidents in Rootly that pull from service impact directly from Backstage, Double click on services in Backstage and browse their respective incident history depending on what was impacted in Rootly. The Bazaar allows engineers and teams to open up and announce their new and exciting projects for transparent cooperation in other parts of larger organizations. Several plugins being developed in a very short time and joining the CNCF were motivating us to not only use Backstage but also contribute and be part of the community. It looks up the app configuration based on the url. In large organizations, engineers often have limited ways of discovering or announcing the projects which could benefit from a wider development effort in terms of different expertise, experiences, and teams spread across the organization. 3b4d8caff6: Adds a new GitHub credentials provider (DefaultGithubCredentialsProvider). And add a permissions field to the PluginEnvironment type in packages/backend/src/types.ts: @backstage/plugin-permission-common and @backstage/plugin-permission-node will need to be installed as dependencies: ## @backstage/plugin-analytics-module-ga@0.1.6, ## @backstage/plugin-apache-airflow@0.1.3. faf844e269: Update @asyncapi/react-component to 1.0.0-next.26, which will fix the This will be remedied in a future update that allows the backend to configured for trusted proxy mode. It is also possible to sort in alphabetical order or on the number of members. Cloud Carbon Footprint is an open-source project, sponsored by Thoughtworks Inc. under theApache License, Version 2.0. Modify your app routes in packages/app/src/App.tsx to include the Bazaar component exported from the plugin, for example: Add a Bazaar icon to the Sidebar to easily access the Bazaar. How was our platform teams journey in using an open-source tool to solve our engineers experience challenges. But like any spreadsheet, some problems started to become more and more evident: In other words, we didnt have confidence in the spreadsheet to be our source of truth about our services and which teams were working on them. It was time to kill it with fire. Anything that is a component used within your company, can be stored along with its metadata. We have laid out steps to set up and run both the frontend and backend plugins within a Backstage created app in our documentation. the possibility to add images. "We're excited to launch this integration of Cloud Carbon Footprint so all developers in the Backstage marketplace are empowered to take actionable steps toward a cleaner future for cloud computing.". Inside QuintoAndar the tools are dynamically provisioned and can have their addresses changed. bdf1419d20: Adds two new home components - CompanyLogo and Toolkit. To be able to accomplish this mission Backstage has some main features. @backstage/plugin-scaffolder-backend-module-cookiecutter@0.1.8, @backstage/plugin-scaffolder-backend@0.15.20. The downside of this change is that secure cookies won't be used unless the backend is directly served with HTTPS. For example: If left empty it will by default use 'Members'. For more details check the following links: da9c59d6e0: Removed @backstage/test-utils dependency. But we didnt stop there, with the entities being created and updated into Backstage we developed some plugins internally to help us track service quality. For that, weve added a webhook in QuintoAndars GitHub organization that sends all push events to a microservice that propagates this event to an Amazon SNS topic and consequently an AWS SQS queue. [2]. a60eb0f0dd: adding new operation to run checks for multiple entities in one request, 48580d0fbb: fix React warning because of missing key prop, @backstage/plugin-tech-insights-common@0.2.1. It was inspired Production-Readiness Checklist Production-Ready Microservices. Below you can see an example of this file: But in order to trust this as a source of truth for our automation, we had to consider: For this, we created an architecture to validate and notify our engineers. Many companies today are of high need to increase the ease of cross-team cooperation. Enterprise Modernization, Platforms and Cloud. Once you have joined a project, you will get access to the community link if it exists. Each project is represented as a card containing its most relevant data to give an overview of the project. Remember the spreadsheet? For example: your projects and its latest activities etc. Automate tedious processes. The Software Template helps a lot in standardizing the creation of services, a simple example is Spotifys Golden Path. It provides a head start to developers improving productivity and dev experience. Accessing our tools wasnt our only challenge. Our Application Security team used Backstage to show vulnerabilities data from various sources within the component tab, giving more visibility into the security aspect of each repository, fostering a culture of security within engineering. An in-depth exploration of enterprise technology and engineering excellence, Keep up to date with the latest business and industry insights for digital leaders, The place for career-building content and tips, and our view on social justice and inclusivity, An opinionated guide to technology frontiers, A model for prioritizing the digital capabilities needed to navigate uncertainty, The business execs' A-Z guide to technology, Bringing the tech-led business changes into focus, Expert insights to help your business grow, Personal perspectives from Thoughtworkers around the globe, Captivating conversations on the latest in business and tech, Learn what life is like as a Thoughtworker, By: Lisa McNally, Head of Cleantech and Sustainability and Tim Cochran, Technical Director. With the possibility of connecting components (applications, repositories, resources) with squad and tribes into Backstage, we were able to give visibility to our engineers about applications ownership, giving them much more transparency. Enable javascript in your browser for better experience. Getting started guide: https://docs.rootly.com/integrations/backstage. Simply put, Backstage is an open-source project created by Spotify and donated to the CNCF (Cloud Native Computing Foundation), which enables us to create a portal for developers. Essentially a marketplace for internal projects suitable for Inner Sourcing. Backstage is a popular service catalog tool open sourced by Spotify and is used by companies such as Netflix, DoorDash, Hello Fresh, and 100's of others including many of our customers. So, as QuintoAndar keeps growing we have a much better tool to address scalability, reliability, and testability issues delivering an amazing user experience to our final users. TechDocs is a feature that brings the docs-like-code idea, which is, bringing the documentation about your service living together with the source code and being accessible by any developer. With that, the Service Catalog was our first objective to bring Backstage into QuintoAndar. Shortcut to create incidents in Rootly via Backstage for easy access, View on-going Rootly incidents in Backstage for broader visibility, Workflow task for Send Slack Message now includes the ability to add an action button for Esclating an incident, Versioning for Rootly postmortem templates, ServiceNow supports custom fields mapping of incident metadata, Workflow task for Sending Slack Message available under Action Item workflows, Ability to handle organizations with thousands of GitHub repositories for efficiently, Schedule Postmortem Meeting task can now exclude weekends, Workflow template for renaming a Slack Channel to a Jira ticket number, PagerDuty and Opsgenie paging tasks in Workflows can now auto-match or select priority levels, Additional Workflow triggers such around creation and update of Postmortem and Causes, Fixed issue where test/training incidents were appearing on the Rootly dashboard and summary emails, Fixed issue on UI where pop-up modules rendered without the background. Although all the main Backstage features have attracted our attention, issues such as documentation and resource creation are challenges that we already have some tools to help our developers in their daily use and that even deserve a dedicated blog post. 10f26e8883: Modify queries to perform better by filtering on sub-queries as well, 1628ca3f49: Fix an issue where the TechDocs sidebar is hidden when the Backstage sidebar is pinned at smaller screen sizes, 2260702efd: Properly exported all referenced types, @backstage/plugin-newrelic-dashboard@0.1.3, @backstage/plugin-tech-insights-backend@0.1.5, @backstage/plugin-code-coverage-backend@0.1.19, @backstage/plugin-search-backend-node@0.4.4, @backstage/plugin-techdocs-backend@0.12.3, @backstage/plugin-tech-insights-backend-module-jsonfc@0.1.5, @backstage/plugin-permission-backend@0.3.0, @backstage/plugin-kubernetes-backend@0.4.3, @backstage/plugin-scaffolder-backend-module-rails@0.2.3. The Bazaar is a place where teams can propose projects for cross-functional team development. As soon as Spotify announced Backstage, we saw that the project had enormous potential and that the proposal makes sense with the challenges we were facing. Ensure that the values that are filled in are valid and that developers would be notified in case of changes to the expected values. Can we find out how many services each squad/tribe is the owner of? With that in mind, our platform teams started to elaborate on the idea of having a portal service so our developers could easily infrastructure and product tools. However, QuintoAndar is a tech company and we want to be part of this growing community. Improve the visualization of our microservices including its ownership, in a way that reflects how our engineering teams were organized. If you use any custom plugins for ESLint please check compatibility. For five days, the teams worked collaboratively to take Cloud Carbon Footprint (CCF) an open source tool that takes advantage of cloud APIs to provide visualizations of estimated carbon emissions on usage across AWS, Google Cloud Platform, and Microsoft Azure and built a plugin for Backstage. Considering our use case, we have a worker that consumes these events and validates if the push was done on the repositorys default branch, which most of the time is the main or master branch, once this push has been done we validate if there is a .checklist.yaml file in that repository. (Ex: APIs error rate, latencies, vulnerabilities), Often the team responsible for that service was out of date, New services were not always inserted in the spreadsheet, Disabled or invalid emails were not updated in the spreadsheet, The repository name was not enough to correlate infrastructure resources to that microservice, New hires were unaware of the spreadsheet. The Bazaar also provides an easy way to manage, catalog, and browse these Inner Sourcing friendly projects and components. This means that addresses that were saved in the favorites bar or in documentations were deprecated with a certain frequency. This plugin brings the functionality of the open source project Cloud Carbon Footprint into Backstage as an officially published plugin on the Backstage Plugin Marketplace. We internally developed a project called Checklist-as-Code to initially solve the main problems mentioned. "As part of Spotify's pathway to net-zero emissions, we leverage Backstage's automation and visibility capabilities to engage our developers in climate advocacy," said Tyson Singer, head of Technology and Platforms at Spotify. Certainly, we have thought about creating our own service and not using an open-source tool to do so. ResizeObserver loop limit exceeded errors which we encountered in several E2E a67ec8527f: Exclude the AWS session token from credential validation, because it's not necessary in this context. This processor reads items from DynamoDB and creates the respective entities in Backstage. For more information on how were working with Spotify and others in the Backstage community to make developers lives easier and improve productivity, please contact us at deveffectiveness@thoughtworks.com. [1]. If the checklist has any validation errors, the workflow makes a comment inside the pull request so that the developer could work on the fix. We had several challenges and we know that we still have a lot to build, improving the developer experience of our developers is a principle for us to scale QuintoAndar in a healthy and efficient way. Using a YAML file as template new services could be created allowing to add the right resources to it. It is a concise summary of all production availability standards and requirements, a set of questions that help to classify whether the service is ready to go into production. These plugins are also open source and open to contributors to constantly improve and evolve! We continue to reduce the fragmentation of tooling by building the most pluggable enterprise-ready incident management platform on the market. Which team should we contact to know about business rules? Browse incident metadata as granular as per service in Backstage such as severity, reporter, status, start/end time, etc. Nowadays these are the main features of Backstage provided at the core, but they have also a list of useful plugins that can be installed individually, which you can find in the Plugins Marketplace. The service assessment is a manual tool where teams can discuss aspects necessary for an application and raise discussions about good software development practices, in addition to proposing ideas of how to improve the application quality. tests. Being able to use Backstage and contribute to It is something that motivates us. When a tech company starts to scale increasing the number of services and engineers, the visualization of resources and tracking of ownership becomes distressed. 3f08dcd696: For the component EntityMembersListCard you can now specify the type of members you have in a group. Using the validator library, we exposed an endpoint that could receive the contents of a file and return if this file is valid checking if it has the fields mandatory and meets the requirements we set. 3b4d8caff6: Allow a GitHubCredentialsProvider to be passed to the GitHub scaffolder tasks actions. When clicking on a Bazaar project a card containing the Bazaar information will show up. Incident management on Slack. "@typescript-eslint/eslint-plugin": "^v4.33.0". The Service Catalog is one of the most important features of Backstage, it is through it that we can store data related to services, libraries, data pipelines. The Bazaar ensures that new Inner Sourcing friendly projects gain visibility through Backstage and a way for interested engineers to show their interest and in the future contribute with their specific skill set. In packages/app/src/components/Root.tsx add: Add a Bazaar card to the overview tab on the packages/app/src/components/catalog/EntityPage.tsx add: The latest modified Bazaar projects are displayed in the Bazaar landing page, located at the Bazaar icon in the sidebar. We have followed this approach so that other automation can consume these push events and perform automation from them. Increasing observability possibilities on e2e tests, Making all the engineering tools easily accessible/updated for everyone. 34883f5c9e: Added possibility to pass customized title and description for the scorecards instead of using hardcoded ones. With "Inner Sourcing", we mean projects that are developed internally within a company, but with Open Source best practices. Connecting QuintoAndar engineering to this developer portal framework will certainly help us to solve more and more challenges ahead. So, using Backstage to catalog our infrastructure made it easy to solve the spreadsheet and ownership problems. So, using organization-workflows we could run a single GitHub Action for our entire organization. The high volume of new services has brought to us concerns about dealing with ownership: This information is valuable not only for the product teams to organize themselves but also for the platform teams to be able to create feasible cost management and incident management practices in our engineering structure. Thinking about how to solve the problems mentioned, our team started to look for a tool that can be a good fit to not only solve the issues but also empower the software development journey at QuintoAndar. 3423b3b24d: Enhance token description by highlighting that the trailing colon is required. We followed the project closely and saw the astronomical growth of the community and how much Spotify itself invested to make the project grow. ## @backstage/plugin-badges-backend@0.1.15, ## @backstage/plugin-bazaar-backend@0.1.6, ## @backstage/plugin-catalog-backend-module-ldap@0.3.9, ## @backstage/plugin-catalog-backend-module-msgraph@0.2.12, ## @backstage/plugin-catalog-import@0.7.9, ## @backstage/plugin-catalog-react@0.6.11, ## @backstage/plugin-code-coverage@0.1.22, ## @backstage/plugin-code-coverage-backend@0.1.19, ## @backstage/plugin-config-schema@0.1.18, ## @backstage/plugin-cost-insights@0.11.17, ## @backstage/plugin-explore-react@0.0.10, ## @backstage/plugin-git-release-manager@0.3.8, ## @backstage/plugin-github-actions@0.4.31, ## @backstage/plugin-github-deployments@0.1.26, ## @backstage/plugin-gitops-profiles@0.3.13, ## @backstage/plugin-graphql-backend@0.1.11, ## @backstage/plugin-kafka-backend@0.2.14, ## @backstage/plugin-kubernetes-backend@0.4.3, ## @backstage/plugin-newrelic-dashboard@0.1.3. https://docs.rootly.com/integrations/backstage. Thinking about the Service Catalog, we knew the complexity and the effort that we would have to invest to develop something completely from scratch. Making developers lives better using yaml Site Reliability Engineer @QuintoAndar, Integrating Lighthouse to your Test Automation Framework, Synchronization and Critical Section Problem, An overview of our open source projects for MACH & Composable, Cypress + Prometheus. Template Software is a mechanism that allows the creation of components inside Backstage directly through the user interface with a few clicks. Now that we can trust the ownership data, we aim to use it to achieve high-quality services creation from scratch. QuintoAndar is a company that uses technology as its main growth driver through several open-source community tools such as Kubernetes, ArgoCD, ArgoWorkflows, Apache Airflow, and Prometheus, we always strive to be close to the community. a2ed2c2d69: - feat: Created PullRequestsDashboardProvider for resolving team and team member relations, feat: Updated useFilterProcessor to provide teamIds for, @backstage/plugin-azure-devops-common@0.2.0, 6eb6e2dc31: Add Bazaar plugin to marketplace and some minor refactoring, 2b19fd2e94: Make sure to avoid accidental data sharing / mutation of, 0fb17da164: chore: bumping dependencies in the GraphQL modules and bringing them up to date with the latest. The image below illustrates this process. name - name of the project on URL safe format, description - present your idea and what skills you are looking for, status - whether or not the project has started, responsible - main contact person of the project, project - link Bazaar project to existing entity in the catalog, community link - link to where the project members can communicate, e.g. Fill Bazaar-tab with more content, for example images and achievements, Show all the members that have joined the project, Extend the dialogue for adding a project with more fields, e.g. From that card it is possible to either link or unlink an entity to a project, edit or delete the project and join the project if it seems interesting to you. Keeping our developers up to date with news is always a challenge and not addressing this issue has a strong impact on engineering productivity. With no good way to find these existing internal projects to join, the possibility of working with Inner Sourcing practices suffers. With Backstages 150+ publicly listed adopters, even more developers now have a new way to gain visibility around the cloud carbon emissions of their projects and to identify concrete actions towards climate goals. Only documenting tools location and how to access them wasnt scalable enough for our scenario. It is typical for developers to use more than one tool in their daily development, such as ArgoCD, Drone CI, Grafana, Thanos, and Kibana. How can we relate microservices metrics to their respective squads and tribes? With this data validated and stored, its finally time for Backstage. Another option is to check which repositories in our organization do not have a. The resulting CCF plugin we created enables developer teams to make strategic decisions that optimize their cloud infrastructure and reduce their carbon footprint. 20ca7cfa5f: Switched the secure cookie mode set on the express-session to use 'auto' rather than true. [3]. Need to know to enable it? And for a long time, the solution to our problem was a spreadsheet . Customers can substitute their Rootly service catalog with their existing Backstage implementation and seamlessly ensure both platforms remain in sync. If youd like to be a part of and help shape awesome processes like the ones described here, join us! This core feature offers a set of resources so that the service documentation written in markdown can be centralized in Backstage. Organizing the engineering structure, its accesses, cataloging resources, and making all the tools accessible to everyone, is a challenge for every company growing in a fast-paced environment. If so, the worker validates if this file is filled correctly, consequently, we aggregate some information from GitHub itself, such as the topics that that repository has, and insert it in an Amazon DynamoDB table. Being part of an open-source community saves us a lot of time not reinventing the wheel, so we can think about what matters: be the destination for housing and conquer the world. We have several squads and tribes within QuintoAndar creating new microservices almost every week. Backstage fully meets our will, the community is bringing great use cases and improving the open-source tool, something that we would hardly be able to do with the same speed internally. To add a project to the bazaar, simply click on the add-project button and fill in the form.