Category Archives: DevOps

On the shoulders of giants

“Our minds are all we have. They are all we have ever had. And they are all we can offer others” – Sam Harris

When we talk about DevOps we often talk about it terms of temporal actions. We talk about culture as an observer or actor, we talk about Automation as a series of tasks, we talk about Measurement as things that we can quantify and we talk about sharing as connecting ideas and expressing them for others to know. Interestingly on top of this we still question or debate or convey what DevOps is or isn’t. (just watch #DevOps on twitter for the ceaseless battle)

To me, DevOps is everything above and much more. It’s an expression of not only empathy for fellow human beings, but an expression of love and passion for one another, not a personal love but a connection to other human beings that transcends in a way that connects us above and beyond what mere words can convey.

We often joke about “Thought Leaders” or express our fear of “Imposter Syndrome” or copy and mimic the perceived behaviors of the “Unicorns” but I find this merely an exercise in humility more than anything else.  All uniformly falling into the not seeing one’s self (or not fully crediting one’s self) as being part of this conscious activity we call DevOps.

This is a “big tent” party, there is no admittance card, just an open mind willing to learn, share and in essence respect one each other as one would love each other – not in a transactional “I love you because..” but in a way that connects the human spirit. It’s okay, not just okay, but perfectly reasonable and human to have your own perspective of what DevOps is and I hope more people feel empowered to share their experiences and their story.

It is this essence I think we need to take our discussion to – for the problems it solves aren’t limited to what we call the “DevOps movement”, it could help in anywhere from education, politics, home, work – you name it – anywhere humans are involved, we can take these lessons learned and apply them (and we should!)

I’ve been pondering how to write something on this for ages..  the last thing I want to do is come off all “hippy dippy” but at the same time, I don’t want to not share my thoughts on this – it’s hard finding a balance until I realized that the balance didn’t really matter. I feel the love, I feel the passion, I feel the energy – I should express this maybe as to remove the cloud of “hippy dippyness” and get down to what really matters. I’m not a psychologist, a scientist or even a philosopher but I am a human being so my opinion / views are merely expressed to share and have a conversation and connect with my readers. You bet I feel a sense of imposter syndrome and at the same time a desire (or internal mission/fire) for ”thought leadering”, but that isn’t what this is about.

The truth is, you really matter. Every single one of you. At first I got into this DevOps game because I felt how it helped, not just where I worked, but how it helped me. Years later, I stick to it and have realized how the lessons/concepts learned aren’t “Development” or “Operations” or even technology related, but universal to the human experience. I’ve shared some of my stories of “applied devops” and “how I’ve applied” it and now I’m sharing how I’ve internalized it and what it means to me – and I believe that is the true value of it therein.

Those of us wanting to share this experience often feel what we call “imposter syndrome” not because we’re faking it until we’re making it – but because we didn’t just invent this up even though we seem like “thought leader” on the subject matter, we’re just connecting/resonating with the philosophical and human experience of DevOps.
It’s this “a-ha” that fired me up to write again.. I feel I should not only explain what my message is in a way but open up to making this a series of how people can connect and relate not just to the experience of DevOps, but as human beings.

What I love about this community is the people. I love it that all the familiar names we know day in and day take time out of their busy lives to connect with us all, to speak to you in the hallway, to connect you to others and share experiences, their lives, their love and their passion. Not as celebrities taking time out of their celebrity lives to appease the plebes but as humans that have a general interest in making the world a better place to be.  The world is so small that even my personal and family experiences connect me with others, I’m warmed at the heart that John Willis and I crossed paths in many more ways than one and it’s those paths that we crossed in our unique ways that gave us a shared experience to know and love – and best of all John is just one of the many figures in this concept that go over and beyond to help make the world a better place.

John’s piece on burnout –  Karōjisatsu is heart wrenching, honest and passionate even if not written from the experiences of a professional psychologist or psychiatrist – but someone showing the humility to help those they love and respect and connect emotionally with other human beings who may be suffering from burnout. It’s hard to write and share such experiences in the light of imposter syndrome and it takes guts of steel to continue to share and talk about this at the many amazing conferences he speaks at and excellent (gut wrenching) stories shared on this subject.

We’re often not experts at all this knowledge, we’re merely connecting with it and expressing it through our own stories – Adding upon it.

“If I have seen further it is by standing on the shoulders of Giants.” – Isaac Newton

On the shoulders of these giants, we learn. We share. We measure. We experiment. We do what we have learned we can do to make our lives, our world, our jobs, our experiences better for us all. When Albert Einstein discovered his theory of Relativity, it was not invented out of thin air, it was a conscious study and internalization of the giants before him and a search for deeper understandings expressed through physics & mathematics. Einstein warrants his contributions to science, physics and the world, there is no doubt – but not as a guilty admittance of imposter syndrome or thought leadering, but as embracing the qualities of the human mind to understand itself and the universe.

We too can (and do) leverage mathematics as the language of measuring and quantifying (and we should get better at it). This is the temporal action of devops, the material result of plan, do, measure act and its super important and well covered in a plethora of other blogs so I’ll say its important, but not really the point. The point is you.

The concepts of DevOps is connecting humans to humans and recognition of our conscious ability to be aware, act on this awareness and respect how each one of us operates, grows and flourishes.  DevOps removes the obscurity of humans getting work done in the frame of “Development” and Operations” and it’s a stepping stone to a better understanding of the human condition so we can remove the influences of bad ideas, failed practices, crackpots and essential mysticisms and philosophies of everything  shaping our way of life and how we get work done.

The point isn’t to convey what we have learned and know as an authority of what should be done, but as a way for us to communicate and share what we know and can know. This is why we don’t say to copy each other and do what works for others – the culture, conscious and soul of a company isn’t universal and can only be understood through introspection and understanding the how and why and removing the barriers of connecting the work to be done through the experience of doing work with other human beings. (which could lend itself to some universality, but that would be a whole ‘nother story.. I don’t think homogenization is the answer 😉 )

We stand on the shoulder of giants recognizing the work of those before us, personalizing the work of today and having empathy for what will come tomorrow – not out of fear of change but the enlightenment of knowing we have the conscious ability to understand and have empathy.

DevOps in essence is the culmination of the human condition, our culture of free (competitive) markets, the popular sharing of ideas and an essence of equanimity as we struggle through our shared experiences and relate to one another. Knowing this helps our internal survival, our cultural survival and our career survival and that’s just the stepping stone of where this movement begins.

—-

As I sit here preparing to click “Publish Post” and I just want to go run and hide in my own conscious safety zone in my mind, but as I grow I learn that what I can share may help others and that sharing is what connects me as a human being and that is what I cherish. Its not the fear of speaking up I should concern myself with, its the fear I could have done something but didn’t that I don’t want.

Advertisements
Tagged

Engineering the future backwards.

I’ll be the first to admit, I’m probably butchering the crap out of what Cognitive Edge put into the “Future backwards” methodology but in butchering the crap out of it, I’ve learned it can be a great vehicle to visualize work but also bridge the communications gap and provide a way to represent emergence, failure, successes, convergence or simply clarify overwhelming decision points.

From Coginitive-edge.comThe Future, Backwards method was created to aid in widening the range of perspectives a group of people can take on understanding their past and the possibilities of their future. The entrained perspectives of people within an organization give them a limited view of the present, and such entrained patterns of past perception can determine its future.

I use the idea of building a future backwards diagram to achieve multiple goals. For me, it helps represent a map of more simple decisions or states and how those states appear to flow from historical perspectives to future perspectives (I try and visualize these forwards and backwards in time and sometimes representative of the decision points made now). Not only to represent as things were, as things are but as things may be and how decision points can lead to future states, surface emergent properties, show how systems adapt & evolve or how things fall back and fade away.

What I love about the ideas behind this future backwards mapping is that it is a visual representation of a complex “thing”. In my talk referenced below I tried to convey that a pterodactyl was misunderstood for YEARS but by visualizing it people were able to convey lessons learned and morph their understanding into a more complete understanding – a pile of bones is simply to incomprehensible of a place to be at to decide what something really is.

Even though this pterodactyl was wrong, it still led to a better understanding of it.

Pterodactylus_antiquus_soemmerring

As for the following diagram that looks like a twisted paperclip – that is a Feynman diagram – a picture that conveyed a very complex interaction of subatomic particles that simplified calculation into manageable chunks and conveyed a way for people to communicate very complex and abstract things.

14265274892_3edf24f774_k

So what is “Future Backwards”?

The future backwards is a method of “sense making” designed/created by cognitive-edge. From Wikipedia “Sensemaking is the process by which people give meaning to experience. While this process has been studied by other disciplines under other names for centuries, the term “sensemaking” has primarily marked three distinct but related research areas since the 1970s: Sensemaking was introduced to Human–computer interaction by PARC researchers Russell, Stefik, Pirolli and Card in 1993, to information science by Brenda Dervin, and to organizational studies by Karl Weick.http://en.wikipedia.org/wiki/Sensemaking

Most of all for *ME* – its a diagram of something complex that I can apply a little science to in order to better understand, measure and predict from.

My Butchered Future-Backwards diagram – Getting Started

long_term_fb

Again, this example image is completely my take on future-backwards and how I’ve used it to make sense of my work. I’ve used it to show an example “Desired” or “future state” and I’ve used it to show our “current state” and then a “concerned state”. The example current state is not positive or negative but the reality of what is driving the business. (nor is this really the objective of the process as described by cognitive edge)

In some cases mapping this out can help you reinforce what you need to succeed. If you need to “keep the lights on” that “current state” may be the most important thing. This was one of the most important things I let escape my mind in my talk about simple diagrams such as this – the visualization of complex parts & decisions that are often lost in translation as teams are making huge leaps into new territory almost as an act of faith rather than a testable hypothesis of “will this work and how can we know”. You may see that you need to distribute work – hand off the “keep the lights on” to an “SRE” team but move the “future” state to an engineering team that can have the focus, budget and resources needed to sustain apparent states.

Emergence

When you map our a “Future backwards” diagram you have done some work on visualizing your work and through these representations you can simplify the calculations and more importantly the communications of what works, what didn’t work and what may have worked better or worse than you expected. You can see if decisions lead to alternate paths or if systems/processes you thought would diminish were actually surfaced and expanded. This process of comparing and looking for variance is how I see adaption, exaptation or even extinction. It’s where I “borrow” another method of setting up some of these decisions / states as “safe to fail” experimentations just to know if they’re viable and surfacing or buffering desired or undesired states & outcomes.

Adaptation / Exaptation

By visualizing your future backwards, you may see patterns that surface all the time or patterns that seem to evolve to always survive even in states you didn’t predict them to be. This is hard to quantify unless you’re mapping them out and you may never recognize such value or in some cases, patterns of failure unless you *do* map it out. Some things can be so well entrenched that you may need to represent them in a future-backwards diagram in much finer granularity than the broad-stroke decision graphs i’ve shown in my examples.

In this example future-backwards map I broke things out a bit more specific to a use case and showed how its less a concern of good/bad/status quote but more a realization of containers, physical machines or virtual machines and possible divergence & convergence routes between the paths.

long_term_fb-2

In the above example a Java shop may choose to do design choices of all of the above or one or the other and you can visualize this to show how two decisions may be similar enough that they converge and the others may be divergent.

I could cite examples all day long, but I just wanted to give more breadth to my talk to try and answer some of the questions that came up and whatever I missed in my 5 minute ignite 🙂

A Lightning talk is a darn near impossibility on this topic, but I hope to one day master it because I think 5 minutes of future-backwards has a payback impossible to to quantify to those who embrace it and butcher it for their own way to improve communications & awareness. Here is my deck from DevOpsDays Austin 2015.

BTW, these diagrams are super simple to make if you have a modern wiki or website/editor with the “gliffy” plugin. Just use the standdard basic shapes and lots of decision triangles. It really only takes a few minutes to do but the payoff can be massive.

I look forward to hearing you’re use of such diagrams and what success/thoughts or feedback you get. (or if it fails, I want to hear that too..)

PS, I apologize @snowded if I’ve butchered your hard work, but it has inspired me to do awesome things, so take that for whatever it’s worth 😀

This is just a way to get you thinking and visualizing. Some people like to do super complex overall “Design sessions” and leave this stuff up to management but as a tech person who works with data, graphs and visualizations all day long it just made sense to bridge this gap not only as finding intent/design in my work but communicating it and formalizing it.

Best of all, when you draw this stuff up and put it up on a wiki, go back to it later and see if what you experience “jives” and make sure to discuss and map what worked/didn’t work to better understand and “make sense” of your own environment. BTW, Adaptation and Exaptation are wonderful things, you can really empower people when you see and foster their creative output and translate that or surface that through future states and its also pretty amazing when something created with different intent can be represented as having value even in places you didn’t expect them to be!

Map/draw/graph on!

Tagged , ,

Austin Puppet User Group – #puppethack Meet-up. Collaborate, Share & Learn.

I would like to invite you to join us at HomeAway for #puppethack, a collaborative, informal online (and onsite for those who register) Puppet community hack day! #puppethack is an opportunity for Puppet community members and Puppet Labs’ employees to tackle interesting or fun Puppet projects.

This is a great opportunity to get hands on with puppet, peer program, share/collaborate and network with people from puppet labs at the same time.

We will have Wifi, Power, whiteboards/post-it-notes & plenty of snacks and an abundant supply of coffee & lunch provided.

Click through the meetup.com site and reserve your spot today!

Puppet #Hackday @ HomeAway – Collaborate, Share & Network with local puppeteers

Thursday, Dec 4, 2014, 10:00 AM

HomeAway North @ Domain
11800 Domain Blvd., Suite 300 Austin, TX

7 Puppeteers Attending

I would like to invite you to join us at HomeAway for #puppethack, a collaborative, informal online Puppet community hack day! #puppethack is an opportunity for Puppet community members (US) and Puppet Labs’ employees to tackle interesting or fun Puppet projects.This is a great opportunity to get hands on with puppet, peer program, share/collabora…

Check out this Meetup →

Tagged ,

#DOES14 Conference Notes

DevOps is a thing in the Enterprise and DevOps Enterprise Summit #DOES14 certainly made the case showing organizations such as Disney, GE, Target and groups within the US Government working on DevOps styled initiatives.

I got home super late and i’m super drained from too much allergy/sinus meds but I wanted to share some thoughts here and express my enthusiasm and gratitude for such a fantastic event.  Expect more soon, but a quick summary follows:

Common aspects of DevOps in the enterprise:

Agile – Lots of Agile transformations and Agile team development. Seemed to be one of the most consistent messages that enterprises feel the need to adopt Agile methodologies to achieve DevOps goals.

Tools – Enterprises aren’t just doing unit tests and artifact management through  CI / CD systems but they’re using them to solve challenges such as code reviews, security, audit, compliance,  risk analysis, security analysis and much more. I was excited to see so much discussion on this topic not just from a developer /operations perspective but how management and auditors see opportunity to drive value here as well.

Case Studies – If you needed/wanted case studies, you simply should have been here. Be sure to watch the #DOES14 website and youtube for release of speaker decks and Videos.

Some common concerns that stuck out were definitely:

User Acceptance Testing – This issue was almost universal regardless if your app is in house or a commercial off the shelf system (COTS) – now that you can empower your development and operations team to automate much of the grunt work of CI / CD and development testing the “UAT” part of QA is still very manual & hands on setting the takt time for your entire organization.

Audits – Lots of concerns/questions on Audit. There may need to be an entire day track to cover the issues of Enterprise Audits.  Maybe next year? We didn’t even scratch the surface on this topic and yet, it was an open question with so many speakers and attendees curious as to what others do here.

Personal Thoughts:

To be honest, I was impressed with the depth and breadth that some of these organizations have adopted Lean / Agile and DevOps values.  If Gene Kim accomplished anything with this conference it was making a bold statement that DevOps is in the Enterprise and backing that up with three days of solid evidence for that.

One thing I hope to see next year is how more organizations derive the value and do value stream mapping. I love the enthusiasm and love seeing what people have done but sometimes I felt people were too enthusiastic about trying to do “DevOps” to be like others that do “DevOps” and spend so much to measure how they compare against others in that regard. I’m not sure that this competitive notion of who can DevOp better than everyone else is wrong per se, but I do feel sometimes people do things to lead an idea more than they do things to solve problems.  A local optimization gone mad if you will 😉   (In a way I have a fear of Applied DevOps..  hehe)

I’m still recovering and getting all of my notes together. If you have any specific questions/comments or if you were at the conference and want to discuss particular topics, get in touch!  More posts to come, especially when I can link in videos / slides to share as they get released/published!

Edit:  Update at 3:18 PM – We discussed the conference a bit on HangOps this afternoon.  Feel free to watch the discussion on YouTube below!

Tagged , , ,

DevOps in the Enterprise – Oracle Financials “Flow”

Is your eBusiness suite environment full of multiple development instances all running their own mini projects of sorts with unique efforts and no alignment?  Do you catch yourself thinking that you need more environments because you have more projects that need to go on? Are you adding more projects to the mix because you’re stuffing your projects with fill time and dependencies and seeing that as idle time to do more?

Lets talk about FLOW.

In my experience, the typical Oracle Financials flow is chaos, managed chaos. Its heavily based upon very inflexible and highly managed plans and projects. These plans and projects often have layers of people management, process management, requirements, expectations, deadlines, goals and ambitions that are based upon assumptions and waste – processes that add no value to our end goal of delivering or supporting the financials applications stack. Come to think of it, I’ve been in this industry for 10+ years and its always been a struggle against unrecognized waste! We build our procedures on waste without really knowing it.. almost habitually!

15 Years of eBusiness suite deployments and the bulk of my work has been waste.

Let that settle in. (I feel anxious just thinking about it!)

I’m not here to point fingers, lay blame or say it was all wrong, I just need this to sink in before you can move to understanding flow, understanding your own waste and understand what it means to operate with “DevOps” value or “lean” operations. We can’t possibly work towards DevOps patterns in Enterprise systems until you realize what your own “Flow” of work is and what waste really is.

So about this waste, lets get into more details so you understand why its called “waste”.

Waste

Warning: This is mostly thoughts flowing out of my head.. maybe not the most cohesive.. most grammatically or syntactically correct, but just thoughts & experiences & ideas. I hope these get you thinking about your “waste” more than they are meant to document all waste.

Learned helplessness – Procedures are in place to trigger punishment, there is no reward for stopping production to do things right, there is no value consensus in the project or collaboration & reflection on getting things done.  Someone else has planned things out according to someone’s idea of a deliverable that someone else forecast and the core value sold to the business it that the project will be done by a specified time and budget. Where is the value in this?

Overproduction – The tendency of projects to keep people busy by losing focus on what the customer has requirements for.  In general over production is producing goods for which there are no orders, but in an Enterprise IT shop, this could translate to having too many environments operating that doesn’t fit your main flow essentially overproducing in order to keep the appearance of utilization.  This also contains Excess Inventory, adding new expensive instances that you then have to have capital appropriations for because the storage, CPU and memory is expensive.  You start passing those costs on to your customers or absorbing them as cost of doing the project / business. You start building Unnecessary movement with too many flows of work because now you have to maintain, patch and clone these environments on disparate schedules thus, creating more slippage in projects, more waste time in cycles.  All of which adds up to Big Buffers,  Defects, and Unused Employee Creativity.  I’ll let those speak for themselves 🙂

With poor flow you build insurmountable technical debt;  we start choosing the “least” concerning option, the “least” risky option, the “safest” route – and generally, not even based upon a consensus but someone actually signing off on the risk as a personal responsibility or PUSHING responsibility up/down the “People who are paid enough” chain. I quote all these “safe” and “least” routes because they’re often done on assumptions and learned behaviors more than they are chosen on  facts and the discussions thereof.

How then do you then start to approach your eBusiness suite flow?

The conceptual practices that people generally speak of in DevOps circles stem from the works of Kevin Behr, Gene Kim & George Spafford (and many.. many others!) through the metaphor of the Three ways they hint of in the Phoenix Project.  I say metaphors because conceptualizing these is fairly simple, but applying them is a different story and they don’t really “Speak” to the typical “Enterprise” shop. (if you ask me..) – BTW, let me be clear, this is mostly paraphrasing the Three ways!

  • The First Way – Create a fast flow of work that moves from development into IT.
  • The Second Way – Amplify feedback loops, so we can fix quality at the source and avoid rework.
  • The Third Way – Create a culture that simultaneously fosters experimentation, learning from failures, and understanding that repetition and practice are the prerequisites to mastery.

I believe the approach to thoughtfully applying and understanding these three ways successfully is to learn from Operations Management, Lean processes and to borrow heavily from lean JIT manufacturing and the Toyota Production System (TPS).

How does Lean workflow improve our flow?

Here are seven traits of lean operations, made possible by well studied and observed lean practitioners..

  1. Builds in Quality
  2. Flexibility
  3. Higher Productivity
  4. Frees up floor space
  5. Improves Safety
  6. Improves Morale
  7. Reduces cost of inventory

Did you read those and ponder on them a bit? Did you think about them in your own environment? I wanted to get them out there to get you thinking about them so the next part makes more sense.. It can be confusing to talk principles, culture and concepts when you don’t feel any bearing to them yourself.

Developing lean / DevOps values & principles.

Principle 1.  Long term philosophy even at the expense of short term goals. If your goal is to generate value to your customers then you can’t achieve that by taking short cuts to appease near term goals. I had a big paragraph describing concepts here, but deleted it because this will make more sense as you get through the rest of the principles. Simply put, your financial application investment is a HUGE long term investment, don’t put it at risk for near term gain! Keep that in mind as you build your values & culture!

Principle 2. Create continuous process flow to bring problems to the surface.  Those project plans, are they full of hand-offs, waits & idle time? (Waste?) Make flow evident in your organization so the project itself is improved iteratively to optimize its own value.  In a continuous flow, you don’t promote your problems to the next “Cell” for them to work on, you surface them to fix them!

Principle 3. Use Pull systems to avoid overproduction Provide your customers with what they want, when they want it and in the amount they want.  Minimize your WIP (Work in Progress/Process) so you can be responsive to day-by-day shifts in demand rather than relying on schedules.  Too much WIP and you start losing your flow and spiral out of control.

Principle 4. level out the workload Eliminate waste, eliminate overburden.  Remove the painful “start & stop” of implementations – strive to work through to completion smaller, more iterative processes.  The more you start and stop work because you have so many inter-dependencies, the more you’re breaking up the workload, the more you’re adding waste & re-work. Projects & WIP generally get out of hand if you don’t value the leveling out of workloads!

Principle 5. Build a culture of stopping to fix problems – get quality right the first time. Don’t build up technical debt! The quality of service we provide is our value proposition to our customers and the business.  Use quality assurance methods available to help get quality right.  This is one of the areas where I’ll mention tools as you can build a process around tools to remove complexity, remove dependencies (decouple!), remove guess work and improve the flow.  Continuous integration environments, Unit testing, Desired State Configs. These are all concepts that embrace the culture of getting quality right the first time and embracing a culture of stopping to fix problems.  “Prod” is no longer just the production instance of your financials application, it is the entire flow!

Principle 6. Standardize tasks Build foundations for continuous improvement and employee empowerment.  Use stable, repeatable methods everywhere to maintain the predictability, regular timing and output of your processes.  Those “controls” you have in place that you thought did this, actually get in the way.  Rethink those!  The tasks are more important than the controls because the tasks empower people to improve them, collaborate on them, test them and analyze them. Obviously I’m not saying to throw out controls, companies have them in place for Audit, legislative, legal and financial accountability reasons, but use them wisely.  Look at your values and see how your “controls” are part of the value chain!

Principle 7. Use visual controls so no problems are hidden. Many people recommend Kanban style “pull” visuals so you can visually represent the flow, but you can expand these into providing visual data across your flows. We often call them “KPI’s”, we call them “Dashboards”,  We call them “Portals”, They’re representations of flow!  Don’t hide problems by keeping people unaware from them, shoving them off to other groups or pushing the blame around, keep them front and center so you know what you have to apply your efforts on!

Principle 8. Use reliable and thoroughly tested technology that serve your people and processes. Isn’t this the expectation we have in buying and paying for Oracle eBusiness Suite? We often imply reliability and thoroughness by forcing long lead times, forcing compliance, forcing validation, forcing checkpoints and buffering time in our projects. We’re doing it wrong!  Build up reliable OS platforms, so you can do OS patching without your Business Analyst having to approve something they have no bearing on. (You know, those change requests that get promoted to 5 different people!).  Build up reliable infrastructure so you can have alerting, collect metrics, analyze data and visualize your flow.  You want to have the trust to do things right as well as the trust to be able to fix things that break. If you’re not innovating, its time to review your process!

Principle 9 – Grow leaders who thoroughly understand the work, live the philosophy and teach it to others. I personally believe that growing an agile / lean / DevOps culture is much more powerful than buying one. The best way to grow people is to empower them and the best way to empower them is to motivate them by making them part of the process, showing them how they contribute to the process and reflecting their value.

Principle 10. Develop exceptional people and teams who follow your company’s philosophy What is your goal? Are you working with these values? Are you looking to “iterate to innovate”?   Do you have a policy of “If you’re not innovating, its time to review your process”? Are you understanding these values? Are you developing them in your teams & company culture?

Principle 11. Respect your extended network of partners and suppliers by challenging them and helping them improve Work with your partners, help them understand your requirements and your flow. Make sure Oracle knows your patterns, make sure your OS vendor knows your strategy, make sure you’re choosing people who support YOUR business.  Don’t let “best practices” or other buzz phrases get in the way. Your best practices are the practices you foster, you measure, you analyze, you validate, you test and you improve. Validate what your partners are doing through the same process you validate your own organization!

Principle 12. Go and see for your self to thoroughly understand the situation! Don’t leave people hanging! Simple as that. If there is a problem, go and discuss it. If something breaks, you should go and stop the flow, see the problem for yourself, get the right people involved to fix it and take ownership.

Principle 13. Make decisions slowly by consensuses, thoroughly considering all options; implement decisions rapidly. If you ask me, this is the best principle ever because its a complete reversal of experience I’ve had where there is very little discussion on doing this well.  Reverse your patterns by striving for excellence in design & implementation, increase your flow so the implementation of decisions is rapid! Often times people assume a project manager has done all of this in advance and the entire project is implementation, that seems entirely backwards and counter productive to me but a trap I see all to often! Make sure stakeholders are holding their stake! 😀

Principal 14. Become a learning organization through relentless reflection and continuous improvement This is it folks. The culture you develop should be based on relentless reflection and continuous improvement. This means experimentation, A/B Analysis, trial and error. This means blameless postmortems. This means collaboration. This means cross domain participation.  This is “DevOps” folks!

What do these principals do for your flow?

These principals help you develop new values and new learned behaviors.  They really have clarity when you tie them back to those seven traits I bullet pointed earlier.

Builds in Quality – Your principals create a work ethic based on quality products to your customer.  Smaller batch sizes, lower risk, automated testing, easier to verify. You’re actually increasing your throughput even through a single flow!

Flexibility – Reduced lead times, faster deployments, faster fixes.  You’re now empowering people to be creative, flexible because the process is creative.

Higher Productivity – Single flow means that the work is getting done and done right. You have less over production, less environment/instances, less coordination, less process management, more focus on agility and very little non value add work.

Frees up floor space – While we don’t typically have “floor space” in the manufacturing sense, we do have “floor” space in the technology.  Single flow means less VM sprawl, server sprawl, less capital tied up in idle hardware. We’re no local focusing on local optimums of optimizing infrastructure use and VM use by using MORE of it to make it appear utilized.

Improves Safety – People usually don’t get hurt physically in eBusiness suite deployments and upgrades but safety can be distilled in many ways. Safety can be seen as stability, Safety can be seen as respect,  partnerships.  Safety can be the protocol by which you solve the insatiable appetite of managers to strive for consistency.

Improves Morale – People are focused on single flows, people feel empowered to make a difference, people are challenged to do better, people feel the reward of a job well done.  People appreciate adding value and creating value!

Reduces cost of inventory – Just like freeing up floor space, you can reduce sprawl, hardware, processes and controls, reducing the cost of inventory, reducing the cost of WIP and reducing the cost of your flow. You don’t need to have environments on standby, don’t need to have excess capacity just in case.  You can use these excess capacities for other purposes that add value!  Use them to experiment, test, verify. Use them for proof of concepts.  The idle time you measure should be idle time you allocate to improvement. If there is one thing you want to stock up on, its an excess of pride, excess of trust, excess of ownership. The ROI on that is through the roof!

The Three ways

Once you start understanding the metaphors, values and principals of lean operations you can start building your own metaphor for the “Three ways”.  Understand your flow,  make feedback part of your process and wrap it all up with culture!

My Story

If after reading my opening statements you feel I’m being snarky in a way, it’s probably because I am.  Not only have I ran into exceedingly costly and ever failing efforts and projects but as a person in this failure loop, its hurt me personally.  Its built up bad behaviors, bad experiences, bad attitudes and so much more!  Poor flow projects turn collaboration inside out, people don’t want to talk, they don’t want to work together, blame takes over, people fight for the wrong types of responsibility and the demands on your technical staff to pull of thankless miracle after miracle become detrimental to not only their attitudes, but their behaviors and even their health. We start striving for the wrong goals, we start striving for deadlines and dates, we hide risk behind the successes of deadlines, we build technical debt not creating a process that allows us to improve our deficiencies and do things “The right way” and when we say the “right way” – that doesn’t mean “My way as I see fit” but the “Collaborative way with continuous improvement in mind so that we know we’re all working towards creating and adding value”.

The end result of determining your flow isn’t just a beautiful “Self-healing”, “self-maturing”,”reflective” project and infrastructure – it’s a cultural shift for your organization that is downright empowering!

I don’t have all the answers, but what I wanted to do was offer an honest critique of where we are failing and ways that we can fix it, building upon the successful values of others. Isn’t that really where our flow stems from? I didn’t invent this, don’t claim to be a master of it, and i’m always having to practice it and think about it.  I believe in sharing it and writing about it since that is part of my philosophy and how I learn as well!

As always, comments/feedback is HIGHLY appreciated! I know this was a long one so thanks for staying with it!

References:

The Phoenix Project

The Toyota Way

Taiichi Ohno

Too many great resources to list, but that is of course, a great start!

DevOps in the Enterprise – Oracle Financials

This post is more of an introduction to a journey of sorts to get thinking about DevOps in an Oracle eBusiness (Financials/CRM) type infrastructure and I plan on developing these posts over time with more details; incorporating feedback on the goal, designs, components and concerns and just as importantly, I hope to collaborate on these ideas to incorporate non Oracle tools.  It’s extremely important (at least to me!) to leverage the existing tools whenever and however possible so that Oracle doesn’t remain a perpetual silo no one else wants to touch!

Back story about me: I started out supporting Oracle eBusiness Suite in the late 90s, running 10.7 NCA, 11.0.3, every release of 11.5.x and currently supporting a 12.1 environment while trying to plan ahead to R12.2.  I’ve supported Oracle eBusiness on AIX, HPUX, Linux, Windows and Solaris environments over the years – through custom processes, oracle best practices to integrating with ITIL tools (Mercury Interactive at the time). I even spent a while working for Oracle themselves supporting implementations for higher education & healthcare verticals. I’ve seen it, I’ve done it, I’ve broken it, I’ve fixed it, and I have a lot of stories to share about it 🙂

Lets get the wheels spinning!

Goal:

Achieve a process of continuous integration (or more aptly “continuous validation”) within the eBusiness suite platform to achieve DevOps value objectives.

Concerns: (a few of many..)

Just how do you do DevOps with eBusiness suite (and similar COTS systems)?

  • How do you unit test?
  • How do you load test?
  • Do you push code to production?
  • How do you provide developers access?
  • What does the process look and feel like?
  • What about Business Analysts?
  • Audits?
  • Performance?
  • Availability?
  • feedback loops!

Components:

The components I use to build my “Enterprise Devops Pattern” for eBusiness Suite.

  • Oracle eBusiness Suite
  • Oracle RDBMS
  • Oracle iAS/Weblogic
  • Oracle Application Testing Suite
  • Oracle Cloud Control
  • Oracle Linux
  • Puppet, PuppetDB & Foreman
  • VMware

Processes:

What are the common processes we can leverage, optimize and built trust upon to greatly reduce  systems complexity?

  • Patching
  • Project work
  • Customizations
  • Cloning
  • Automation
  • Interfaces
  • Integrations
  • ….

Putting it together

How do we build a system that embraces trust, is reliable and performant? How do we test and prove our value stream?

  • Measuring
  • Experimenting
  • Audit defense
  • Monitoring.. testing..
  • Feedback loops
  • Reporting..

As you can see, just by starting to list out bullet points, the complexity of the eBusiness suite starts to rear its ugly head.  One quickly realizes that while the eBusiness suite is comprised of hundreds of connected “Applications” (AR, AP, GL, CRM, and FA so on and so forth) it really is its own “system” that Oracle has built and bolted on to over the years.  By thinking of it in terms of systems, we can start to see how we can apply DevOps values into the system to solve issues that plague the system by and large – Complexity, Mean times to do anything,  long project cycles, operational silos, skill silos, deep reliance on vendor support and so much more.

I hope to see people join me on this journey, share their ideas & experiences and challenge my assumptions.  The end goal is absolutely a “cookbook” of ideas to bridge the Oracle Financials “monstrosity” (for lack of better word) into a DevOps value stream.

In opening the proverbial flood gates, I want to speak to you – the people implementing, supporting, planning, patching eBusiness suite and see what you are doing. I want to hear from Vendors working on solutions, from groups developing custom solutions to open source projects that can be used to help provide the tools for change and help consolidate some of this data to report back and share.

So please, contact me through my Blog,  LinkedIn, Twitter, Google Plus, e-mail or phone.  Let’s talk! BTW, This work will be shared here and on the DevOps Enterprise Patterns group I’ve volunteered to assist Gene Kim with as well. Please join us there!

Links:

DevOps Enterprise Patterns

Devops Audit Defense Toolkit

Contact Me!

Tagged , , , , , ,

Phoenix Project, the alternate Brent universe

I’ve read, and re-read the Phoenix Project and absolutely love it, learning something new each time and picking up new ideas to keep my brainstorming going, but one thought that has plagued me through to the end every time, is the “guardian angel” to management and the IT person staying the IT person.

How different would the story be if Brent was the benefactor of Erik’s advice?

In my years of experience, I often find the Brents of the industry aren’t wholly self-made but often easily fall into traps that we’re not taught to get out of.  It’s easy for management to rely on us, keeping us as the choke point of many critical projects without realizing it or facing it and we burn out, get angry, go BOFH.  We’re often managed into systems, managed into context and managed into a story and we often try and logic reason ourselves out of this with no end in sight. What if there was a different story?

How different would the Phoenix Project be if Brent was able to learn some leadership, learn some collaboration and develop his persuasion skills to help motivate and transform the organization and himself from within?  Would this story better help others facing similar situations? Would it help develop more leadership roles from the Brent types over the world? Would it help many of us Brent types “see the Forrest from the trees” more clearly?

The book is very good, don’t get me wrong, but I just have a gut feeling every time I read it that I wish Brent could have been developed, fostered and nurtured, not as a follower of management making changes but as the catalyst for management making changes and as a growth path for Brent himself, because I feel that better parallels my experience.  Obviously in the story Brent does benefit from the changes and we can all learn from the story, but in the end, it feels like a management win and it left me wondering how I could do something as if I’d just meet Erik.

I parallel this thought to many of the discussions I see on twitter and my own sorted experiences to help make changes at an organization.  In some tweets, I see people struggling to do what I’d call applied DevOps, replying to others to quit a job because they can’t convince management/peers of the logic of DevOps, people describing you must to this, do that, and run this or be that, and lots of frustration or indifference towards DevOps values by everyone else but them.  I’ve just been insanely curious about trying to parallel the story of the Phoenix Project to myself, my own short comings, my own experiences and those I observe (such as the mentioned tweets and indifference) and how to approach this. It was an awaking of sorts after reading MANY stories and books on leadership and growth that not only are a lot of “Brents” trying to make these changes in their organizations  but ironically they do so in a very “Brent” way  and here is how I’d answer that story.

For me, the new story steams from a few books I’ve read (mentioned below) and an excellent class available from the Teaching Company – Transformational Leadership and how I thought this could make a new Story for Brent if Brent meet Erik and hand the skills to transform.

How to sell DevOps at your org if Brent was the benefactor of Erik’s advice.
The hard way – The Brent way of convincing organizations of devops.

  • Strongly stated position – speak to things as facts – applied “DevOps” – get focused on mimicking successful organizations more than anything else. We’re good at having strong views and strongly stated positions.
  • Assertive Supporting Arguments – “This is the way, we must do this or else…” – Our logic is what we feel makes us assertive and how we speak objectively to something as if that will win people over.
  • Closing the deal – resisting compromise, using only logic/data and extreme passion to make out point.  Combining everything above, we often trap ourselves into an all or none type system/belief.

Would the book have made much more sense to many of us if Brent was developed to learn better collaboration? Be a better leader?  Would you be able to recognize the hard way above as the hard way? After all, most leaders are taught that the better way of persuasion (and leadership) is, collaborative persuasion. Collaborative persuasion helps us achieve the very goals we convey and yet, we don’t speak or really practice to this much, if any at all. While I felt the leadership story in Phoenix Project was very mature and bold, I’d just like to see more parallels of how Brent could have been fostered to lead the transition too and I believe many of us do it the hard way, or Brent way.

The Collaborative Way – Collaborative persuasion – using the very concept we’re trying to convey as a solution as the solution.

  • Establish Credibility – Don’t over estimate your own self!  (Big thing for us Brent types!) – Conduct experiments & develop pilots to share what we’re trying to sell. Build credibility as DevOps applies to YOUR organization. Begin Somewhere!
    Build TRUST! Would Brent be able to experiment  (Vs jump to applying) with the advice of Erik? Could we translate that to Management/Leadership? Could we do so without falling back to our logical bias and do it collaborative for small wins and tactical leadership positions? Automation may be Brent’s goal idealistically (it is after all, following our logical trend) but could we step back and be happy showing a Kanban process and the wins thereof or simply starting with communication? Could we lead the small wins to a strong catalyst for cultural changes? Sometimes culture defies logic and that’s tough for us Brent types to understand or often don’t comprehend at all.
  • Framing for common ground – It’s important to collaborate, it’s important to explain things and frame them in such a way to express ourselves better.  Instead of saying “this must be the way, look, everyone else is succeeding at it” frame the topic so that the focus is drawn to the attention of where you desire to be. Lead people to positive results.  Framing is how you can steer the DevOps story to be better applicable and connected to your organization.
  • Connect emotionally – Is this possible for Brents of the world? Can we adopt our strategy to a broad audience in a collaborative way that connects with management and works through these gatekeepers? Can you focus on the divergent thinking to make it happen? Are we presenting REAL solutions or are we presenting problems? I know it’s terribly easy for us Brents to focus on problems and solutions as if their logically black and white. Can you connect emotionally to your peers and speak to them as if you put on their shoes?
  • Evidence – how can we build stories, examples and metaphors?  How can we make this evidence connect emotionally? Frame it for common ground? Build trust and establish credibility?  Evidence absolutely comes natural for many of us “logical Brents” but the hard part is connecting it to the collaborative persuasion process and realizing that this collaborative persuasion it was grows us from a Brent who may have learned something and wants to achieve it into a Brent who becomes a leader to enact it.

Brent types can transform to be leaders. As leaders, we don’t need formality to express our desires, we need strategy, history, allies, solutions and we need to work with and through the gatekeepers to tell the DevOps story. I think once we learn this, we also learn that perhaps there isn’t an absolute way for everything, that collaboration can lead to new ideas, new thoughts and new models of getting things done. We can not only grow to become leaders, but grow in our understanding and interpretations of the values of DevOps and how those values benefit the growth of your organizations and just as much, if not more importantly the growth of ourselves.

So while the Phoenix Project didn’t focus on the Brent solution as many of us are or have been over the years, I hope this post invigorates you, Brent or not, to think differently, think like a leader and realize that what you are trying to achieve doesn’t have to be an effort of management directly, but an effort that when done right can come directly from you. I implore you twitters not to tell people to quit their job, give up, find something else to do or beg and chew your way up or simply try and mimic or apply logic as if that’s the only way –  as DevOps values aren’t something we believe in and force upon others, they’re  stories that connect emotionally and speak elegantly to us to learn, practice, share and collaborate on. We need to do better selling that story to management & leaders or better develop ourselves to become those management & leaders.

Thanks for your time.  I realize not every Brent wants to be a leader and may be quite content with an organization that delivers DevOps values for him/her, but at the same time, I think there are a lot of us who do want more and I hope this helps you think of how to achieve more and how to lead that effort in a very DevOps way.

Books & Resources I’ve drawn much of this from and wouldn’t want to do without 🙂

PS, I’m terribly new to writing, feel free to leave any feedback about my style, grammar, spelling or any methods to improve.  I’m mostly sharing my stories and thoughts but I’m always open to improve.  Thanks!

As always, feedback appreciated.  Feel free to find me on twitter as well.

Tagged , , , , ,

Devops – It’s about critical thinking & the evolutionary “WHY” of Silos.

I believe one of the best things to ever come out of DevOps movement that no one seems to be describing is essentially an explosion of critical thinking and reasoning skills, The maturing of IT if you will. Some people prescribe different views or distill it into different methods such as C.A.M.S (Culture, Automation, Measure, Share), some people say it’s the destruction of silos, some say it’s just a cool cultural shift and an awakening of sorts. I’d like to take you on a short journey and focus on two things for this topic – the concept of critical thinking and silos.

I found this blog post by Ben Kepes VERY interesting, where he discusses the odd effect of DevOps teams becoming silo teams in and of themselves and how that could appear to be negative or wrong. While I think it is an interesting point to bring up, I think its made without looking at the “WHY” it happens and more of a conceptual view of “WHY” it shouldn’t happen. So lets put on our critical thinking caps and “think here for a minute” about the whys of silos 🙂

From my perspective, the biggest reasons we see silos aren’t vanishing (and quite possibly are being replaced by new silos) is that the silos are the result of real differences of domain. By this I mean, the domain knowledge of your operations, development, analysts, QA, DBA’s, sysadmins, engineers or any group are essentially domains of expertise. When it comes to cultural changes, the best change we can do isn’t to squish these silos down and pretend they don’t exist, but to align them better and give them strategic significance. If a silo really is a moving target, lets align them so they’re all moving in the same direction!

It seems to me, we’re more focused on prescribing ideas rather than deriving solutions. We see improvements because we aligned developers and ops into a team and we think “wow, getting rid of those silos worked” but maybe the silos weren’t the problem to begin with, maybe it was a failure of alignment, workflow, poor cooperation and poor business decisions. By aligning people and connecting them to not only costs of doing business but the revenue of doing business the entire “WIP” chain can see the impact of their work as a whole, as a unit, and as a team (Something to Measure!). Much like the comparisons of WIP from an IT shop to WIP of a manufacturing floor as described in Gene Kim’s excellent book “The Phoenix Project” we have to remind ourselves that even though the mfr floor looks like its running on its own and well oiled and doing small and simple tasks – those small and simple tasks are not just the experience of repetition, but silos of domain knowledge and experience that you don’t NEED to squish and quite frankly, do better when left to be developed and fostered.

I had my first real world experience of this in the early 2000s when I worked at a manufacturing plant. I was a technical lead for implementing an upgraded ERP/MRP system and one day, the manufacturing union went on strike and we full-time employees were led out into the floor to run the plant to finish delivery of critical items. I never thought much about it other than how interesting of an experience it was to go through a labor strike and realize how specialized some of the manufacturing jobs are and how they can appear simple but be rather complex in the end.. I realized I couldn’t assemble tubes like they did, my ceramic painting skills were terrible and the output/workflow of my experience was disastrous at best. Again, after reading “The Phoenix Project” I had a few epiphanies and differing views of the “flattening of silos” that I just couldn’t figure out how to express for a while and it’s just barely coming out here. It dawned on me that the people working this assembly line have a domain knowledge of their own. The people fixing the assembly line have a domain knowledge of their own. When the line breaks, no one expects the line workers to come in during their off hours and fix it because they need to be bothered to learn something about what broke, The line maintenance and facilities crews do their thing. Both domain experts of workers and repairers also have their pipeline, their WIP, their process improvement and they coordinate and match these skills, not distill them and flatten them. So why have such an expectation in IT?

So long story short, I believe we need to do better at understanding the “silos” we complain about. Again, the existence of silos in and of themselves doesn’t mean that they’re all silos of control, silos of despair & silos of differences and if they are these described silos, well, there lies your problem. You can fix the flows, you can align the groups and you can integrate teams and get them working together without artificially ignoring domain knowledge the same way you can prevent domain knowledge creating artificial silos.

The “Why” of Silos isn’t necessarily a bad thing by itself. Have you ever put your critical thinking cap on and thought about YOUR personal “silos”? Are you able to be “culturally adept” but still hold on to personal prejudices against all things commercial or all things not invented here? Do you let your hatred of specific vendors / ideas / practices tunnel vision your beliefs? Are you using silos as a big stick to ignore domain knowledge and experience? Are we really being critical thinkers?

Think critically about your organization. Understand how important your role in IT is to the business and its bottom line. You’re all on the same team, and there may be many silos, but as long as all the silos are on the same “farm”, they really shouldn’t be a problem. In fact, you may soon realize that when your silos are there exactly when and where you need them and this may be more strategic and important than tearing them down and having everything scattered across the field.. metaphorically speaking 🙂

We evolve into silos of domain expertise as a natural progression of our careers, aspirations and experiences. These silos can be very powerful if used wisely and correctly recognized and aligned by your organization. Perhaps DevOps is its own silo, especially if you imply DevOps with specific ideas, tools, solutions and services and that’s why I think silos are less important than we realize and the real concept we should be focusing on is “critical thinking” because it’s that skill that really matters. It’s critical thinking that will give you the gut feeling of what works and what doesn’t and where you need to align yourself and your peers within your organization.

Hope to write down more thoughts on this later.. what do you think?

Tagged , , , , ,
%d bloggers like this: