Layer7 API Management

  • 1.  Multi Team Development

    Posted Apr 20, 2015 10:15 AM

    Hello,

     

    I'd like to reach out to the community to understand how teams are using the API Management platform.  We have a number of teams that are going engage with the platform.  I'm not finding a lot of material on how to structure things to support development by multiple teams. Here is brain dump of what I'm considering:

     

    • Establish a multi-tenant environment where there is a portal/gateway pair for development and testing
      • Use host or port to distinguish environments for development and testing
      • Create a policy hooked to message-received
      • Create necessary policies to route (including portal + gateway interactions)
      • Designate folders that are to hold policies for each environment
    • Enable and create API Owner Groups so that each team can work in their own space, limiting the noise from other teams
    • Designate API owners per team who are able to create and manage APIs for their groups
    • Since we have a multi-tenant situation, there needs to be some sort of convention put in place to distinguish environments (really wish the portal supported better filtering!)
    • On the gateway side... Create and permission folders per environment.  I'd like to permission folders such that only certain users can create objects in designated areas (really wish the portal understood the gateway's folder structure!)
    • Create API groups and bundle development, testing APIs into a group for the consumer to bind to (maybe use port to distinguish between environments?)
    • Allow consumers to register on the portal, create application requests, and get API keys
    • I'm still looking for a reasonable way to manage documentation (portal CMS is proving to be inadequate)

     

    I'm interested in providing a good experience to our development teams from both an API provider and consumer perspective (in our case, our API producers and consumers are internal).  I'd be very interested in hearing how other organizations are doing this.  I'd also be interested in any resources that discuss this topic.

     

    Thanks!

     

    Alejandro



  • 2.  Re: Multi Team Development

    Posted Apr 23, 2015 10:41 AM

    Anyone able to assist with the above question?

     

    Thank you



  • 3.  Re: Multi Team Development

    Posted Apr 28, 2015 07:13 PM

    Alejandro,

     

    It looks like you're on the right track:

    • use API Owner Groups to publish and manage the APIs for each group (you'll want to set the APIs to private to ensure they're exposed to the correct users)
    • optionally use API Groups to create a group of all the APIs that belong to each group
    • create an Account Plan and assign APIs (or a group of APIs) to it
    • assign each of your development groups to the correct Account Plan

     

    That should get you to the point where multiple development groups are working with and managing their own APIs on the Portal.

     

    You mention that you're "still looking for a reasonable way to manage documentation." What issues are you running into? Some suggestions:

    • you can add more info to the auto-generated API doc page by editing the header and/or footer
    • you can version the API doc with the API, maintaining a parent-daughter(s) relationship of pages to show how the API has evolved


  • 4.  Re: Multi Team Development

    Posted Oct 06, 2015 12:21 PM

    Alejandro,

     

    How far did you get with this?

     

    We are in the process of implementing something similar.

     

    We have hundreds, if not thousands of developers. The developers belong to different organizations, and application teams within those organizations. We are trying to leverage the RBAC capabilities of the products to be able to assign as granular of permissions as possible for each team.

     

    I am just curious if you have any lessons learned, limitations that you have identified, or specific things that worked or did not work.

     

    Thanks,

     

    Josh



  • 5.  Re: Multi Team Development

    Posted Oct 14, 2015 09:56 AM

    Hi Josh,

     

    Still rolling this out.  I have a number of developers, but not all of them know policy development.  To me, that is a little bit dangerous.  Especially when we consider the deployment scenarios and the tooling around Policy Manager and the Gateway.  The Portal just adds another layer of complexity.  I'm in the middle of putting together development material for our teams.

     

    My original post started out with a virtualized environment.  While this is possible, it's not where I landed.  I chose to put a Gateway instance in all pre-production environments - DEV, QA, and UAT.  I chose to have a Portal only in UAT (and of course production).  This allowed me to treat the migration of a service to UAT and PROD as an "event".  API key and plan enforcement are really necessary at this stage in the game, but I can go without in the early environments.

     

    Here is what I'm kicking around for my development process. Feedback is welcomed!

     

    • A small group developers will be responsible for API policy development.  This isn't just service policy, but all shared "include" fragments, cluster props, etc...  Basically, a small group needs to "own" development on the Gateway, a kind of "center of excellence" model.
    • Policy development will be done using Policy Manager.
    • Developers will keep policy in a folder structure on the Gateway where each service is under its own top-level folder.  This might become unwieldy if there are a number of services developed, but we'll cross that bridge when we come to it.  Folders are an easy container for policy and security.  My thinking is that we'll be able to migrate folders in and out using name.
    • We will use version control to house Gateway artifacts captured under a given folder.  We use SVN.  We use Crucible for reviews.
    • Using GMU, developers will migrateOut their folder and store a "bundle" xml artifact, a mapping file, and properties files for each environment.  This assumes that the policy is templatized to take advantage of environmental differences.  A couple of notes:
      • Given the state of the deployment tools (GMU), someone needs to have intimate knowledge of how to read and interpret the impact of the changes they are migrating.
      • Someone needs to examine the bundle.xml to determine what dependencies are moving and if they are going to damage anything in the target environment
      • Build automation will use source control as a source for deployment artifacts and automation scripts will be used to migrate artifacts across environments.  I'm considering Maven or ANT to do this, but shell scripts will work just fine.
      • Services need to be marked as "new or update" so that changes are applied properly.
      • Shared include fragments will live in their own space and can migrate on their own.
    • I want to establish a migration account that our automation scripts can use.  I can then lock down folders if there is a need to do so based on developer roles (department, line of business, etc...).  Right now, I don't see the need though.
    • Portal - Since we're dealing with an API Portal in our higher environments, we need to be sensitive to object ids.  It looks like portal related policies are tired to the service's object id. If these get out of whack, bad things happen.  This is important from the deployment perspective. 
    • I'd prefer to establish portal managed services in early environments (development), and then move then through to the higher environments as they are tested and approved.  Ultimately, I want to move the same service to production.
    • Once a service hits a Portal managed gateway, it is enabled using the Portal interface and added to whatever plans need it.  We are geared towards internal development, so we have one Org and Plan to deal with.  But, the process would be the same if we expanded this out further.
    • Rollbacks are a bit tricky…  There are a couple of rollback scenarios to deal with
      • Simply rolling back policy based on source is the simplest and most common approach.  If dependencies are managed correctly (using mapping rules), this should be done using the prior "tag".
      • It may be necessary to snapshot the environment (we're using VM's) and restore the snapshot if things go south.  This may be needed if there is a global change that breaks everything.  Snapshots are simple to take but eat of disk. 
    • SVN - Here is the SVN structure that I'm kicking around:

     

    SVNROOT/APIM-Policies/

    /trunk

    /branches

    /branches/service1/dev   (unstable development work)

    /branches/service1/stable (stable, tested, work)

    /branches/service1/features (feature branches for in-flight development work)

    /tags

    /service1-build#

     

    Development is always done on the dev branch of a policy and then merged into stable. Build automation promotes tags cut from stable.  I considered having one top level project for each service, but that is a lot of overhead.  I'd rather look in one place to see all policy source.

     

    Rant...  The GMU product REALLY needs to have a graphical tool built around handling mapping files.  I can envision some type of interface that allows someone to list ALL dependencies of an artifact (folder in my case).  From there, they can flip the mapping action associated with each object, then generate the mapping file.  This would cut some of the mapping file interpretation that is now needed to properly build out a mapping file.  Additionally, there could be a set of "default" mappings that could be declared so that certain artifacts are always handled the same way.  If I get free cycles, I might take a stab at building something.  At the very least, I can pass this along to the product team...

     

    I plan on putting together a development guide for our teams this week.  There will be gaps, but it will be a start.  I'd be interested in more input from the community around this.  Everyone does this differently, so there is no one sized approach.  But I feel like policy can (and should) be treated like code and follow similar patterns.  From my perspective, I see the Gateway along the same lines as a shared database where everyone is mucking with database objects.  It's not the best setup from an isolation perspective, but it can be reasonably tamed.

     

    Thanks,

     

    Alejandro



  • 6.  Re: Multi Team Development

    Posted Jun 21, 2016 11:24 AM

    Hey Alejandro,

     

    We are just starting with the usage of CA API gateway. Thank you for sharing thoughts. 

     

    Is zoning a strategic direction that could be helpful in this context if you would consider multi team development ?

     

    Regards,

    Erik



  • 7.  Re: Multi Team Development

    Posted Aug 09, 2016 11:03 PM

    Hi Alejandro,

     

    Thank you for sharing this in the community. Can i check with you how it goes currently and if you would able to share with us how the export to the folder can be done via GMU? We are currently designing the flow of build and release management flow for the CA Layer 7 API gateway 9.1 with source control management GIT.

     

    Thanks!

     

    Regards,

    Jason



  • 8.  Re: Multi Team Development

    Posted Nov 17, 2016 08:28 AM

    As for now we successfully massively deployed around 50 appliances on our 3 datacenters (EMEA region, APAC and AMER), distributed in 4 environments (Sandbox, Development, Testing, Production) with both DMZ and internal LAN functions, load-balanced.

    Due to this number of appliances (software version), we've developed a framework enabling us to interact with API Gateways using shell-level commands, covering nearly all Restman APIs available.

    One example being able to deploy/update with a single click the whole organization tree (>15 developers communities) with associated roles, permissions, folders, security zones, and generic administrative accounts for those communities; on all appliances.

     

       > On the gateway side... Create and permission folders per environment.  I'd like to permission folders such that only

       > certain users can create objects in designated areas (really wish the portal understood the gateway's folder structure!)

     

    This is exactly what we've done.

    - dedicated folder per developer community

    - read visibility for shared policy fragments folders (ex: OTK and MAG, + company wide shared policies).

    - allowing create/read/update/delete for cluster-wide properties, restricted to objects with names starting with <community_name>.<project>

    - allowing create users to onboard developers

    - allow creation of private keys, certificates, etc, segregated by name prefix, so that one community can't see nor interact with objects belonging to other communities.

     

       > Allow consumers to register on the portal, create application requests, and get API keys

     

    We're now struggling with Portal and API Owner Groups in order to segregate communities, but have trouble handling APIs created on the Policy Manager as they show up as "Pubic" on the Portal, and due to API Owner Group feature enabled, no one but the Admin can see them. Still working on this, maybe we'll open a case.

     

    As per the migration, we're using SVN and GMU embedded into an in-house developed automatic migration tool.