Profile photo of admin

by admin

ManageIQ Design Summit Preview – Come Fleece with Us

October 3, 2014 in Syndicated


Next week is the first ever ManageIQ Design Summit, and the agenda looks great – a real opportunity to learn more about cloud management in general, and how we’re shapping the roadmap of ManageIQ in particular. Here are some of the highlights of what you’ll see:

Event Kick-off – October 7, 9am

Oleg Barenboim will discuss where we are, engineering-wise, and then Xavier Lecauchois, CloudForms product czar, and Jarid Cottrell, Booz Allen Hamilton architect, will talk about the state of cloud management and where ManageIQ fits.

Extend Track

There are two tracks in the design summit, extend and design. The extend track is all about how you can utilize ManageIQ, color outside the lines, and push it beyond the scope of what it does “out of the box”.

John Hardy and Brad Ascar give Tour de Force of ManageIQ

If you’ve ever been curious about how to do more with ManageIQ and automate All of the Things, the first day will be a wonderful experience for you. From utilizing state machines and navigating the ManageIQ object model to mastering the powerful policy engine and using ManageIQ as a RESTful backend platform, these talks are key to understanding how to tailor ManageIQ for your environment. For additional information on these topics, see their recent podcast.

Creating a Cloud Services Broker on ManageIQ – Booz Allen Hamilton

Jarid Cottrell and Nirmal Mehta, whom you may have met on the ManageIQ podcast, (notice a trend?) will give an overview of their cloud broker and how they built it on ManageIQ. Their project is a testament to the power and versatility of ManageIQ as a platform.

Extending ManageIQ’s IPAM Feature Set – Cox Automotive

JD Calder and Jason Cornell will discuss how they modified ManageIQ to fulfill their self-service requirements. And yes, JD also appeared on the podcast.

Design

The design track will consist of developers plotting out the next release – and beyond. If you want to push the project into the future, or you just need to know, this track is for you.

Modularization Direction and Roadmap – Oleg and Jason Frey

As of now, ManageIQ is a single monolithic release, but that may not be the case in future releases. You can hear about some of this in Jason’s podcast.

Heat/CloudFormation Integration – Greg McCullough, Bill Wei

Orchestration is big – and essential to future development. This talk will focus on two of the main technologies for cloud automation and how to interface with them.

Plugin Provider Architecture – Greg Blomquist, Jason Frey

One important thing on the roadmap is the ability to more easily add in new service providers. At the moment, adding a new provider is manual and non-trivial.

Cloud Broker Integration Roadmap – Jarid and Nirmal

Jarid and Nirmal talk about how to integrate future versions of their cloud broker project with the ManageIQ object model and UI.

Upgrading to Ruby 2.x / Upgrading to Rails 4.x – Joe Rafaniello, Aaron Patterson

These are actually two separate sessions separated by lunch. ManageIQ is a Rails application, but it is built on outdated Ruby and Rails architectures. Time to figure out how to upgrade them. Aaron gave us a preview of what the work entails in his podcast.

Hacking Room

If you’re inspired to start a project as you’re surrounded by the talented folks in the community, we will supply a breakout room for you to do just that on Wednesday, October 8. Get some developers and enough caffeine to do the job and do something cool.

Be there!

These are just the highlights – there’s more where that came from. Take a look at the agenda and see what you can see. Enjoy the show!

Profile photo of admin

by admin

Podcast with JD Calder, Cox Automotive

October 3, 2014 in Syndicated

Cox Automotive (nee Autotrader Group) is a founding member of the ManageIQ Community. JD Calder talked about their contributions and what you can look forward to at the Design Summit.

Cox Automotive has been a long-time ManageIQ/CloudForms user and even hacker, modifying pieces to build an IPAM for their service catalog.

This will be the final design summit podcast! Register now, because this will be quite the show.

Profile photo of admin

by admin

Podcast 4 with Jarid and Nirmal from Booz Allen

October 2, 2014 in Syndicated

Jarid and Nirmal brought their late night act to the podcast, talking up their cloud broker software, open source in the cloud, and the role ManageIQ played in helping them tie it all together.

Booz Allen Hamilton is a founding member of the ManageIQ community, a co-sponsor of the design summit, and is doing some amazing things, integration-wise, with ManageIQ. Look for some great sessions from them next week discussing their upcoming open source release.

Profile photo of admin

by admin

Jason Frey Podcast – Design Summit, Fleecing, Security, and Plugability

October 2, 2014 in Syndicated

Jason Frey is a long-time contributor and architect for the ManageIQ and CloudForms world. If you want to know anything about ManageIQ, he’s your guy. Watch and learn from the following video.

After talking with Jason, you begin to understand just how wide-ranging and comprehensive ManageIQ is.

You’ll get to meet Jason and many other smart people at the ManageIQ Design Summit, coming up on October 7 & 8 in Mahwah, New Jersey.

Profile photo of admin

by admin

Open source more about process than licensing

October 2, 2014 in Syndicated

It is a testament to the success of the Open Source Initiative’s (OSI) branding campaign for open source software that “open source” and “licensing” are functionally synonymous. To the extent that people are familiar with open source software, it is the source code released under a license that lets anyone see the “crown jewels” of a software program as opposed to an opaque binary, or black box that hides its underpinnings.

This well-trodden trope has dominated the mainstream view of open source software since Eric Raymond pushed it into the public consciousness over 15 years ago. But taking a previously proprietary code base and transitioning it to an open source project makes one seriously question any previous assumptions about code and licensing. It is that undertaking that leads one to appreciate the values of process and governance. After seeing that transition from closed to open firsthand, I am convinced that the choice of whether to release code as a proprietary or open source project leads to fundamental changes in the end product, a divergence that is very difficult to roll back.

From the point of view of most people, the software license is the most important aspect of releasing open source software, but in my opinion, licensing falls somewhere under user experience, workflows, and integration into existing data center technologies. Nowhere is this difference, in what is “known” (licensing) and what is the actual reality (user workflows), more clear than in the fearful eyes of the development team tasked with transforming their proprietary product into an open source project. In fact, the development methodology chosen by the engineers has a direct impact on what type of software is produced. If an open source development model is chosen from the beginning, one can be reasonably sure that the end product will be relatively portable and will plug into the most commonly used environments. If a proprietary model is chosen, it’s very easy for the developers to make cheap shortcuts that result in short-term gain and long-term pain—and that’s precisely what often happens.

To the extent that people think of these things, the common perception is that this change involves a simple search and replace, maybe the removal of 3rd party software, uploading to a public repository, and presto! Fork me on GitHub! But, nothing could be further from the truth. What most people miss about software is that it’s much more about process, control, and administration than software licenses. As I argued in It Was Never About Innovation, the key to the success of open source software is not the desire for innovation but rather the fact that all players in open source ecosystems are on a level playing field. Customers, outside developers, freeloaders—they all have a seat at the table and can exert influence on a project by virtue of their leveraging of community equity, which they have built up over time by contributing in various ways. This is in stark contrast to proprietary development models where developers can essentially do whatever they want as long as they create an end product that meets the expectations of the Product Requirements Document (PRD) supplied by product management.

This is where the difference between open source and proprietary development comes into stark relief. The open process that accompanies open source development will help to ensure that the software will likely integrate into any given environment and that some bad habits are often avoided. These two things go hand-in-hand. For example, proprietary software development often results in software that is monolithic in nature with a minimum of dependencies on system software and often bundled with its own set of libraries and tools. This gives developers the leeway to do whatever they want, often employing specific versions of libraries, reinventing various wheels, and generally veering far from the path of creating software that works well in a broader context.

Open source software developers, by contrast, have no such luxury. From day one, their users demand the ultimate in flexibility, integration, and conformance to standard data center systems practices. This means the utilization of existing tools and libraries whenever possible, baking into the process the idea that your software will be a cog in a much larger data center machine. Note that nowhere did I mention that open source development was faster or more innovative, although it can be. On one hand, developers love the fact that they have complete control over the end product and don’t have to deal with annoyances, such as customer demands that their precious software honor their existing workflows. On the other hand, end users love the fact that their open source deployments likely have a long history of use within large data centers and that those previous users made sure the software was to their liking.

Both of these approaches come at a cost: open source development may actually be slower at particular times in its life-cycle due to some overhead costs that are inherent to the model, and proprietary development, while perhaps faster, sends the developer team down the road of maintenance hell, needing to endlessly maintain the bits of glue that generally come for free in open source development. The overwhelming evidence of late suggests that the open source approach is far more effective in the data center.

Suppose that your team went down the road of proprietary development but eventually came to the conclusion that they could win over more users with an open source approach—what then? Here lies the conundrum: the process of undoing the proprietary process and imbuing a project with the open source sauce is spectacularly difficult. Many otherwise knowledgeable people in the tech industry have no idea just how much change is involved. Hell, most engineers have no idea what’s actually involved in switching horses midstream. To engage in the process means necessarily losing valuable development time while taking up tasks that developers feel are, frankly, beneath them. To change software from a monolithic, proprietary code base to one that plays well with others is a gargantuan task.

“But wait!,” I can hear you say. “Can’t they just release whatever they have under an open source license and then take care of the other stuff later?” Sure, they can, but the end result will likely be disappointing at best, and a colossal disaster at worst. For starters, mere mortals won’t be able to even install the software, much less build it from source. There are several tricks developers play to make black box monolithic products work for their end users that make it terrible for open source community-building:

  • Highly customized build environment and tools. This is the #1 reason why the majority of proprietary software cannot simply be set loose as open source: it’s completely unusable to all except the developer team that built it. When developing open source software, there are a few standard ways to build software. All of them are terrible at producing highly optimized executable programs for running at the highest level of efficiency, but they’re great for giving developers a simple, standardized way to build and distribute software. The process of making your proprietary software build with standardized open source build tools is probably non-trivial. Open source projects, by contrast, came out of the crib compiling with GCC.

  • 3rd party libraries, also proprietary, that you do not have permission to include in your open source code. Even if your code can build with GNU autotools and GCC, to use one example, you probably have to rewrite some not-insignificant portion of the code. This takes time and effort away from your developers who will be spending time ripping and replacing many pieces of code and not implementing new features. This varies from project to project, but it afflicts the vast majority of projects going from closed to open.

  • Bad security practices. When developers think nobody else is looking, they do all sorts of crazy things. And as long as features are developed on schedule, nobody bats a eye. It is this primacy of feature development over code quality that can result in some horrendous security holes. Obvious exceptions aside, *cough*heartbleed*cough*, there is lots of evidence that open source software is more secure than its proprietary counterparts.

  • Bad coding practices and magical unicorn libraries. For the same reasons as above, ie. feature primacy and nobody’s looking, developers tend to work with the latest and greatest from other software packages, especially when it comes to runtime scripting engines, libraries, and tools. They take the code, modify it, and then they have an end product that works. For now. This is great if you’re on a deadline and your code must work by midnight, and it’s approaching 23:30. The problem, however, is that the product will live long after midnight tonight, and you will be responsible for maintaining, updating and syncing your pristine unicorn library with code that will inevitably diverge from what you modified. This is terrible for everyone, developers and admins included. Imagine the poor sod in operations assigned to installing and maintaining someone’s late-night “innovations”.

All of the above leads product teams to one obvious conclusion: package and distribute the software in such a way that it runs as far removed as possible from the system on which it resides, usually in the form of a bloated virtual appliance or at least in the form of a self-contained application that relies on the bare minimum of system libraries. Windows admins should take a look at their Program Files directory sometime. Or better yet, don’t. All of this, taken together, adds up to an end product that is extremely difficult to release as open source software.

Some ops people might think that an appliance is easier for them to deploy and maintain, but more often, they hold their nose in order to use the thing. They will tolerate such an approach if the software actually makes their jobs easier, but they won’t like it. All of the ops people I know, and I used to be one, prefer that the software they deploy conform to their existing processes and workflows, not force them to create new ones.

Put another way: would your software exist in its current form if it started life as an open source project? Or would end users have demanded a different approach?

Open source is about process much more than license, and everyone in an open source community has the ability to influence those processes. Projects that start out as open source have many characteristics baked in from the beginning that often, though not always, save developers from their own worst instincts. If you elect to reverse course and move to the open source model, understand what this change entails—it is a minefield, laden with challenges that will be new to your development team, who are unaccustomed to seeing their practices challenged, don’t particularly relish direct customer feedback, and are entirely uncomfortable with the idea of others reading over their shoulder as they write code. The amount of effort to change from proprietary to open source processes is probably on the same order as going from waterfall to agile development.

Example: ManageIQ

When Red Hat acquired ManageIQ in late 2012, it was with the understanding that the code would be open sourced—eventually. However, there were several things standing in the way of that:

  1. Many of the User Interface (UI) scripts and libraries were proprietary, 3rd party tools.

  2. The software was distributed as an encrypted virtual machine.

  3. ManageIQ was and is a Rails app, and some of the accompanying Ruby gems were modified from their upstream sources to implement some specific features.

#1 meant that many parts of the code, particularly in the UI, had to be ripped out and either replaced with an open source library or rewritten. This took quite a bit of time, but was something that had to be done to release the code.

#2 is not something one can do in an open source project, striking fear into the hearts of the development team. Some changes to the code were necessary after losing the (false) sense of security that came with distributing the software in an encrypted appliance.

#3 meant that the developer team had to carry forward its modifications to custom gems, which was becoming a burdensome chore and would only get worse over time. The developer team is still in the process of fixing this, but I’m happy to report that we’ve hired a strong Ruby developer, Aaron Patterson, who will, among other things, maintain the team’s changes to upstream gems and prevent future forks and divergence. He’ll also lead the effort to convert ManageIQ to Ruby on Rails 4.

Conclusion

Be considerate of your developers and the challenges ahead of them. Hopefully they understand that the needed changes will ultimately result in a better end product. It comes at a price but has its own rewards, too. And never forget to remind folks that choosing an open source approach from the beginning would have obviated this pain.

Lead Image: 
Rating: 

(8 votes)


Add This: 




Channel: 
Article Type: 
Default CC License: 

Profile photo of admin

by admin

Avoid Shellshock and the Bash Plague with ManageIQ

September 29, 2014 in Syndicated

Unless you’ve been under a rock the last few days, you’ve no doubt heard of the Shellshock vulnerability affecting a large number of *nix machines with the Bash shell installed. Note: Bash doesn’t even need to be the default shell – plenty of ‘Dash’ users are also affected. Luckily, there’s a way to avoid this mess – a policy management engine in ManageIQ, combined with VM fleecing, that lets you route around the vulnerability, turning off VMs that are vulnerable. This video gives you the goods:

That video comes courtesy of John Hardy, all-around good guy (mostly) and CloudForms man about town. He wrote up an accompanying blog post that includes the bits needed for implementation:

Fancy finding out really quickly if your [linux boxes] are patched correctly? Even if they are turned off right now? Wow that is clever not even the virtual infrastructure players can do that…I know…its cool. Here it is..

Using Cloudforms (or ManageIQ for FREE!) download this policy and import it into Control. Then assign the policy to your targets. The policy will only check Linux systems, though it could do with a makeover to check only RHEL 6.5 systems too.

Download and import this policy profile (GitHub.com)

Protect yourselves out there!

Profile photo of admin

by admin

From Russia with Tenderlove – ManageIQ Podcast

September 26, 2014 in Syndicated

Aaron Patterson joined us from Russia on this fun podcast, covering many topics, from the Rails 4 migration and the ManageIQ Design Summit to why you should “just use Ruby”.

You can see Aaron and lots of other smart people at the upcoming ManageIQ Design Summit on October 7 & 8 in Mahwah, New Jersey.

Profile photo of admin

by admin

Design Summit Podcast – Brad Ascar and John Hardy

September 24, 2014 in Syndicated

John Hardy and Brad Ascar stopped by the studio to discuss their talks at the upcoming ManageIQ Design Summit on October 7 & 8 in Mahwah, NJ

John and Brad’s upcoming talks:

  • Fisher Price: My First State Machine – John Hardy
  • Navigating the ManageIQ Object Model – Brad Ascar
  • Advanced Policy State Management – Brad Ascar
  • New for Anand: RESTful API – John Hardy

Musical intro: Cantina Rag, by Jackson F. Smith

Profile photo of admin

by admin

Breaking: Tenderlove to Discuss Rails 4 Migration Plans at Design Summit

September 19, 2014 in Syndicated


If you’re headed to the ManageIQ Design Summit on October 7 & 8, you’re in for a treat. Aaron Patterson (@tenderlove) will be leading a session in the design track on the topic of migrating ManageIQ to Ruby on Rails 4.

One of the things Aaron will valliantly tackle, in addition to Rails 4 migration and his general all-around awesomeness, is ensuring that ManageIQ stays in sync with upstream gems. At the design summit you’ll have a chance to hear him talk about these and other topics – essential knowledge for developers.

Profile photo of admin

by admin

Sprint 12 Demo Results and Video

September 18, 2014 in Syndicated

We just finished up the Sprint 12 demo, ended September 9.

If you want to read the highlights of the sprint demo, notes are posted below:

Sprint 12 Demo Highlights

  • Overview
  • Sprint Statistics
  • User Interface
  • Automate
  • Appliance
  • REST API
  • Fleecing

Sprints

  • 3 weeks long each ending on a Monday
  • GitHub Milestone per Sprint
  • GitHub Milestone called Roadmap

Sprint 12 Ended Sept 8

  • links to sprint issues
  • Over 90 Pull Requests Merged!
  • 63 issues labeled as “bug”
  • 15 issues labeled as “appliance”
  • 22 issues labeled as “enhancement”

User Interface

  • Form Buttons (css based)
  • Patternfly modified Bootstrap

Automate

  • EMS Refresh (Amazon virtualization type)
  • Exposed cloud relationship in service models
  • Persistent state data through retries
  • Automate Model changes
  • High level rubyrep changes

Amazon Virtualization Type

Service Models: Cloud Relationship

class Flavor
    expose :ext_management_system
    expose :vms
end

class FloatingIp
    expose :ext_management_system
    expose :vm
    expose :cloud_tenant
end

class SecurityGroup
    expose :ext_management_system
    expose :cloud_network
    expose :cloud_tenant
    expose :firewall_rules
    expose :vms
end
class AvailabilityZone
    expose :ext_management_system
    expose :vms
    expose :vms_and_templates
    expose :cloud_subnets
end

class CloudNetwork
    expose :ext_management_system
    expose :cloud_tenant
    expose :cloud_subnets
    expose :security_groups
    expose :vms
end
class CloudSubnet
    expose :cloud_network
    expose :availability_zone
    expose :vms
end
class EmsCloud
    expose :availability_zones
    expose :cloud_networks
    *   expose :cloud_tenants
    expose :flavors
    expose :floating_ips
    expose :key_pairs
    expose :security_groups
end

Persistent state data through retries

  • New automate methods for state machine methods:
    • state_var_exist?(var_name)
    • set_state_var(var_name, value)
  • get_state_var(var_name)

Example:

if $evm.state_var_exist?(“test_data”)
    test_data = $evm.get_state_var(“test_data”)
    # TODO: Something interesting
else
    # First time through, initialize data
    $evm.set_state_var(“test_data”, 1)
end

Automate Model changes

  • Auto-placement run from a state machine step for Cloud and Infrastructure provisioning
  • Added common “Finished” step to all Automate state machine classes

Added eligible* and set* methods for cloud resources to provision task service model

  • eligible_availability_zones
  • eligible_cloud_networks
  • eligible_cloud_subnets
  • eligible_cloud_tenants
  • eligible_floating_ip_addresses
  • eligible_guest_access_key_pairs
  • eligible_instance_types
  • eligible_security_groups

Automate changes

Console

  • Config temp disk for OpenStack Fleecing
  • Key generation

Security

  • CertMonger Integration
  • IPA Research for Single Sign-On
  • Appliance

Appliance

  • Ruby 2.0 changes (compatible with 1.9.3)
  • Logrotate now rotates our logs!
  • Gem upgrades for bugs/enhancements
  • haml
  • net-ldap
  • net-ping
  • Added/labelled issues for ruby2.x/rails4.x

REST API

  • Update for Authentication
    • With External Authentication (httpd) enabled against an IPA Server
    • fixed bug in the REST API and Appliance Console to honor the external credentials like the Web UI when targeting the /api entrypoint

VM Fleecing

XFS Filesystem Support

  • Work in Progress
    • see PR – https://github.com/ManageIQ/manageiq/pull/490
  • Most of the code is complete
  • Lacking Unit Tests
  • Expected to be completed in Sprint 13