Accompanying repository for the "Testing and Integration (The Remix)" talk given at CraftConf 2015. Feel free to open any issues for questions and/or to say hi :)
Let's face it, testing distributed systems is hard. Large number of inputs, partial failures, and asynchrony make these systems seemingly impossible to verify. Formal testing methods are onerous and system correctness continues to elude us. Despite the difficulty, we need to have a way to change our systems with confidence. Integration and unit tests are vital to our sanity and we know it's good for us to use continuous integration to ensure functional consistency across our products.
Come to this talk for a fresh and practical look into the various aspects of distributed systems testing and integration. We will cover CI pipelines: their strengths, weaknesses, and ways to improvement them. Get ready to rediscover the untapped power of your integration practices and develop a burning desire to make them more awesome!
See the image credits, link to slides, and video
The talk is broken down in 3 main sessions:
- Testing Distributed Systems
- Challenges of testing
- Distributed System Testing in Academia
- Continuous Integration
- CI challenges
- Patterns
- Conclusions
The epic soundtrack to this talk can be found on the CraftConfTalk Spotify playlist. If you liked this talk and you are interested in Computer Science check out Papers We Love. We have chapters in many cities and aim to bring academic research close to practitioners.
- My RICON 2014 talk has a deeper look into academic testing.
- http://www.mountaingoatsoftware.com/blog/the-forgotten-layer-of-the-test-automation-pyramid
- http://technologyconversations.com/2013/12/24/test-driven-development-tdd-best-practices-using-java-examples-2/
- Canary Tests
- http://martinfowler.com/bliki/TestPyramid.html
- http://google-engtools.blogspot.com/2011/06/testing-at-speed-and-scale-of-google.html
- http://googletesting.blogspot.com/
- Development at Google and slides
- How Google tests software
- Google Test and Automation Conference
- Golang Testing
- Unit Testing in Golang
- TDD in Golang
- Canary Analyze All The Things: How We Learned to Keep Calm and Release Often by Roy Rapoport of Netflix
- http://en.wikipedia.org/wiki/Continuous_integration
- http://martinfowler.com/articles/continuousIntegration.html
- http://www.thoughtworks.com/continuous-integration
- http://www.amazon.com/gp/product/0321336380
- http://www.slant.co/topics/186/compare/~circleci_vs_shippable_vs_codeship
- https://www.quora.com/What-is-the-difference-between-Bamboo-CircleCI-CIsimple-Ship-io-Codeship-Jenkins-Hudson-Semaphoreapp-Shippable-Solano-CI-TravisCI-and-Wercker
- CircleCI Configuration
- TravisCI Getting Started
- TravisCI Common build problems
- Flickr's flipping out blogpost
- http://www.karlgroves.com/2014/03/13/everything-you-know-about-accessibility-testing-is-wrong-part-4/
- http://pivotallabs.com/accessibility-ci-failures/
- Continuous Deployment @ Flickr
- https://www.chef.io/solutions/continuous-delivery/
- ChefConf 2015: Chef Delivery talk & Slides
- https://github.com/opscode-cookbooks/delivery-truck
- https://github.com/chef/delivery-cli
- Package Management in Chef
- On Delivery lumosity blogpost
- Adam Jacob's ChefConf 2015 keynote & https://github.com/chef/devops-kungfu
- ChefConf 2014: Jamie Winsor, "The Berkshelf Vision"
- How does Etsy manage development and operations
- Flickr
- 10+ Deploys Per Day: Dev and Ops Cooperation at Flickr
Thank you to everyone who helped with feedback/resources and advice for this talk. Special thanks to: Caitie McCaffrey, Jordan West, Jon Hyman, Bruce Spang, Devon O’Dell, Kelsey Gilmore-Innis, Aysylu Greenberg, Alan Kasindorf, Paul Reed, André Arko, Mike O’Neill, Thomas Mahoney, Eric Kustarz, Ian Fung, Neha Narula, Karl Smith, and Greg Bako.