Senior Software Engineer - Incident Insights & Readiness

About the position

The Incident Insights & Readiness SRE team at Datadog fosters a resilient culture by using incidents as learning opportunities and catalysts for growth. Our users are Datadog engineers, and we build the software, tooling, and operational frameworks that help them prepare for, respond to, and learn from incidents. We work closely with engineering teams across Datadog to analyze incidents and turn those insights into better tools, stronger incident response, and organizational learning. Our efforts empower Datadog to navigate unexpected failures confidently, efficiently, and with a commitment to continuous learning and systems improvement. At Datadog, we place value in our office culture - the relationships and collaboration it builds and the creativity it brings to the table. We operate as a hybrid workplace to ensure our Datadogs can create a work-life harmony that best fits them.

Responsibilities

  • Own and improve the on-call experience for the company by establishing best practices and building platforms to support on-call rotations and compensation.
  • Define how we respond to incidents, lead the design and implementation of software to streamline the process, and collaborate with product teams to improve incident response across Datadog. Our aim is to fully support our incident responders in dealing with complexity.
  • Contribute to the post-mortem process for the company, collaborating with teams on writing them, and identifying opportunities to reduce friction and enhance learning value for the organization. Our team also runs a weekly postmortem reading group.
  • Support various teams in facilitating incident reviews that emphasize learning and blamelessness. Help them share their learnings across the organization to improve the resilience of our people.
  • Provide technical leadership and day-to-day coaching to team members, accelerating their growth through design reviews, collaborative problem-solving and operational excellence best practices.
  • Train our on-callers in incident and post-mortem processes, sharing expertise in incident management best practices. This involves both introducing newcomers to on-call responsibilities and refreshing the knowledge of existing engineers.
  • Lead cross-functional initiatives in engineering organizations across Datadog, embedding with teams to understand their challenges and drive lasting improvements to reliability and operational excellence.

Requirements

  • At least 5 years of experience building software that solves real user problems. Experience designing new features and collaborating on code and technical design reviews. We primarily develop in Go and Python, with a bit of TypeScript.
  • Experience building or operating distributed systems, with familiarity with Kubernetes and an understanding of complex failure modes.
  • Demonstrated ability to independently own ambiguous technical problems from design through delivery while balancing long-term engineering quality with pragmatic execution.
  • Experience analyzing incidents, identifying systemic risks, and driving engineering improvements informed by operational learnings.
  • Experience participating in on-call rotations and improving incident response processes.
  • Empathy, collaboration, and communication skills in English to cultivate strong relationships across various teams in the organization
  • Experience mentoring engineers, driving cross-functional initiatives, and influencing technical direction without relying on organizational authority.

Nice-to-haves

  • Experience serving as an incident commander or incident coordinator is a plus.

Benefits

  • New hire stock equity (RSUs) and employee stock purchase plan (ESPP)
  • Continuous professional development, product training, and career pathing
  • Intradepartmental mentor and buddy program for in-house networking
  • An inclusive company culture, ability to join our Community Guilds (Datadog employee resource groups)
  • Access to Inclusion Talks, our internal panel discussions
  • Free, global mental health benefits for employees and dependents age 6+
  • Competitive global benefits
  • healthcare
  • dental
  • parental planning
  • mental health benefits
  • a 401(k) plan and match
  • paid time off
  • fitness reimbursements
  • a discounted employee stock purchase plan
Back to blog

Common Interview Questions And Answers

1. HOW DO YOU PLAN YOUR DAY?

This is what this question poses: When do you focus and start working seriously? What are the hours you work optimally? Are you a night owl? A morning bird? Remote teams can be made up of people working on different shifts and around the world, so you won't necessarily be stuck in the 9-5 schedule if it's not for you...

2. HOW DO YOU USE THE DIFFERENT COMMUNICATION TOOLS IN DIFFERENT SITUATIONS?

When you're working on a remote team, there's no way to chat in the hallway between meetings or catch up on the latest project during an office carpool. Therefore, virtual communication will be absolutely essential to get your work done...

3. WHAT IS "WORKING REMOTE" REALLY FOR YOU?

Many people want to work remotely because of the flexibility it allows. You can work anywhere and at any time of the day...

4. WHAT DO YOU NEED IN YOUR PHYSICAL WORKSPACE TO SUCCEED IN YOUR WORK?

With this question, companies are looking to see what equipment they may need to provide you with and to verify how aware you are of what remote working could mean for you physically and logistically...

5. HOW DO YOU PROCESS INFORMATION?

Several years ago, I was working in a team to plan a big event. My supervisor made us all work as a team before the big day. One of our activities has been to find out how each of us processes information...

6. HOW DO YOU MANAGE THE CALENDAR AND THE PROGRAM? WHICH APPLICATIONS / SYSTEM DO YOU USE?

Or you may receive even more specific questions, such as: What's on your calendar? Do you plan blocks of time to do certain types of work? Do you have an open calendar that everyone can see?...

7. HOW DO YOU ORGANIZE FILES, LINKS, AND TABS ON YOUR COMPUTER?

Just like your schedule, how you track files and other information is very important. After all, everything is digital!...

8. HOW TO PRIORITIZE WORK?

The day I watched Marie Forleo's film separating the important from the urgent, my life changed. Not all remote jobs start fast, but most of them are...

9. HOW DO YOU PREPARE FOR A MEETING AND PREPARE A MEETING? WHAT DO YOU SEE HAPPENING DURING THE MEETING?

Just as communication is essential when working remotely, so is organization. Because you won't have those opportunities in the elevator or a casual conversation in the lunchroom, you should take advantage of the little time you have in a video or phone conference...

10. HOW DO YOU USE TECHNOLOGY ON A DAILY BASIS, IN YOUR WORK AND FOR YOUR PLEASURE?

This is a great question because it shows your comfort level with technology, which is very important for a remote worker because you will be working with technology over time...

Other Jobs To Apply

Channel Relations Specialist at Sandvik: 100% Remote (Mebane, NC, US)

Reservations Assistant | Remote | No Experience Needed

UPS Remote Jobs (Data Entry, Part Time) $22/H Hiring Now

Remote RN Telephone Triage - $1000 Sign On Bonus!

Remote Tech Support Representative Entry Level Text Based Only No Degree Required

FT Data Entry Clerk (Work From Home)

Immediate Hire:Flight Attendant(Part-Time)

Technical Support Specialist

[Remote/WFM] Jetblue Airlines Hiring: Data Entr…

Remote Chat Jobs Part Time Evening – No Phone, Set Your Own Hours | $25–$35/hr

Delivery driver | Lewisburg, PA

Work From Home

Mortgage Operations Specialist

Stay-at-Home Moms 21–39: Help bless a family - Prior Birth Required Paid $50K-$85K

Online Remote Work

Senior Engineer - Cybersecurity Operations & Engineering (Remote)

UGC - Create videos at home - No experience required

Customer Service Representative [Work From Home]

Virtual Clerical Assistant; Remote

Part-Time Center Associate Sinking Spring 5606

Part-Time Instructor (Remote) Computer Information System - Online College for Career Advancement

Remote Client Agent (Work From Home) | Flexible Schedule

Production Associate

Remote Customer Experience & Analytics Lead

Remote Customer Experience & Analytics Lead

Remote Response Associate, Roadside Assistance (CSR)

Financial Professional - Remote

Entry-Level Parts Advisor

Shift Manager

Office Staff - Seasonal

Warehouse Worker/Cover Driver

No Experience Necessary | Work from Home Chat Agent Job - $25-$35/hour (Remote), Compensation: $25-$35/hour

Patient Service Representative (Remote – Full Time)

Remote Telehealth Nurse (Nights & Weekends)

Part Time Customer Service Representative (Teleradiology Support)

**Entry-Level Remote Chat Support Specialist – Level 1 – Immediate Start at arenaflex**

[Hiring] Designer, Creative Services @Abnormal

reputed company Clerical Data Entry-Writers Work reputed company

Reservation Agent Part-Time Evenings

Remote Data Entry Assistant for Teens – Flexible After‑School & Weekend Hours, Earn $13/hr with arenaflex

HR Virtual Assistant (Remote - Accra, Ghana only)

**Entry-Level Remote Chat Support Specialist – Level 1 – Immediate Start at arenaflex**

Remote YouTube & TikTok Content Creator for Test Prep

[Hiring] Designer, Creative Services @Abnormal

reputed company Clerical Data Entry-Writers Work reputed company

Remote Night reputed company Chat Customer Service Agent – Earn $25–$35/Hour, No Commute, No Degree Required – Join arenaflex

Reservation Agent Part-Time Evenings

NOW HIRING: EXPERIENCED FREIGHT AGENTS (1099, REMOTE)

Remote Data Entry Assistant for Teens – Flexible After‑School & Weekend Hours, Earn $13/hr with arenaflex

Entry Level Remote Chat Operator (No Experience)