Sep 20 - Sep 25 Wrap-up

Spent most of my week in Portland to attend conferences.

Conferences

• Attended LDAPCon 2009 and published report.

• Attended LinuxCon 2009.

Image Store Proxy

• Updated image-store-proxy to 1.0. This version brings support for gpg signed images. Still need testing against the real-world Canonical Image Store infrastructure.

A summary of LDAPCon 2009

On Sunday, September 20th and Monday, September 21st I attended LDAPCon 2009 in Portland, OR. Most of the open source projects were there - with the notable absence of Port 389 (Redhat) - as well as some vendors (Apple and UnboundID). Most of the slides are available online.

Apache Directory project

The Apache Directory folks gave several presentations:

Apache Directory Server provides an integrated product with most of the standard network services: in addition to ldap, dns, dhcp, ntp and kerberos services can be enabled as part of a deployment. Kerberos support seems to be in early stage as it almost works. Another interesting aspect of the project is its integration with the Eclipse environment. Apache Directory Server is embedded in Apache Directory Studio. The latter provides a management tool for Directory administrator. If the Eclipse integration in Ubuntu is improved Apache Directory Studio would be a very good addition to the archive.

An overview of implementing replication in the Apache Directory Server project was given. RFC 4533 is used as the basis for LDAP replication in OpenLDAP. The goal here was to be able to replicate between Apache Directory Server and OpenLDAP. This may be the start to a standard replication protocol between different directory products.

Three components needed to be implemented:

• the consumer part is the easiest and can be a standalone component. It receives LDAP entries updates and can do whatever it wants with them. It reminds me of similar requests I heard at the MySQL User Conference last April where people were interested in having an easier access to the MySQL replication log.

• the producer is more complex to implement as it requires to keep a log of the modifications done on the server.

• conflict resolution is the hardest part and mandatory if multi-master is to be supported. The Apache Directory Server decided to implement a strategy of last writer wins as they're trying to not require any user intervention for conflict resolution. I'm not convinced this is the best strategy though.

While implementing replication support they've also added support for store procedures and triggers.

LSC Project: LDAP Synchronization Connector

Corporate environments usually have multiple identity repositories and keeping all of them in sync can be quite a challenge. The LSC project aims at automating the task of keeping all identity stores up-to-date. Written in java it can read and write to any database or LDAP directory. On-the-fly transformation of data sources are possible and the framework tries to make it easy to implement new synchronisation policies.

Another great tool that could be added to the directory administrator toolbox to help integrate Ubuntu in existing infrastructures.

Storing LDAP Data in MySQL Cluster (OpenLDAP and OpenDS)

This was a joined presentation between the OpenLDAP and OpenDS projects. A new backend has been added to store entries using the MySQL Cluster NDB API. The main advantage is to be able to access the same data over SQL and LDAP as well as providing a highly-available infrastructure with data distributed on multiple nodes. Both OpenDS and OpenLDAP have worked together to create a common data model highlighting that cooperation does happen in the LDAP space.

A Panel discussion among the representatives of the various LDAP Projects on roadmaps

Sunday ended up with a panel where representatives of different directory vendors answered questions from the audience. Each open source project briefly outlined a few points they were trying to improve: documentation for OpenLDAP, data migration for Apache Directory and multiple schema support for OpenDS. The issue of virtual directories was also discussed with the need of more GUIs to cover administration tools as well as workflows. Apache Directory Studio was given as a potential good starting point to build these higher level tools. The subject of standard ACL's was also covered. It seems that this is still a sensitive issue in the community and projects are still arguing about a common solution. One option put forward was to look at the X500 ACL model and start from there.

The last item of discussion covered how to expand the user base of directories. The world of directories is rather small and its use cases are usually associated with Identity Management (User and Group, Authentication). Having good client APIs was mentioned as an option. However the whole group ran out of ideas quickly and got kind of stuck in front of this problem.

Directory Standardization Status

Directory standardization happens within two bodies: X500 in ISO/IEC and LDAP in IETF. The most important topic currently discussed in both bodies is password policies. A new draft of an IETF document is being worked on by Howard Chu and Ludovic Poitou.

Other topics being worked on cover:

• Internationalization (with Unicode support in LDAPprep and SASLprep)

• simple LDAP Transactions (to cover adding entries to different containers)

• replacing DIGEST-MD5 with SCRAM

• vCard support

On the front of Directory Application schemas support for NFSv4 Federated Filesystem and an Information Model for Kerberos are currently being worked on with drafts available for review.

The question of starting a new LDAP working group within the IETF was raised. Topics that could be covered include:

• LDAP Chaining Operation


• Access controls: based on the X.500 model with extensibility added.


• LDIF update


• LDAP Sync/ LDAP Sync-based Replication


• Complex Transactions


• Password Policies


• Directory views


• Schema versioning

LDAP in the java world

LDAP support in java is being actively worked on especially on the SDK front. OpenDS, Apache Directory Server and UnboundID have released new open-sourced SDKs to improve the aging JNDI and Netscape java SDKs. All of them are rather low-level implementations. The three projects are also working together to find a common ground.

There is also some progress made at the persistence level. The DataNucleus project gave an overview of adding LDAP support to the standard JDO interface. The goal is to provide a reference implementation of JDO for an LDAP data store.

Unified Authentication Service in OpenLDAP

Howard Chu gave an overview of the new modules developed in OpenLDAP related user authentication. Based on the work from nss-ldapd the nssov overlay provides integration with the pam stack as well as the nss stack. Disconnected mode in the pcache overlay has been added in the latest version of openldap as discussed during the Ubuntu Developer Summit last May. Most of this work is already available in Ubuntu Karmic and improvements should be made during the Lucid release cycle.

Another interesting module is the integrated certification authority. If a search request for the userCertificate and userKey attributes for an entry is made and these attributes don't exist they're generated on the fly. This should help out in creating an X.509 base PKI.

LDAP Innovations in the OpenDS project

The last session of the conference was given by Ludovic Poitou of the OpenDS project. New features available in OpenDS include tasks as well as extended syntax rules. Time matching rules have also been added so that queries like "give me entries that have a last login time older than 3 weeks" can be expressed directly in ldap and processed by the server. That brought some interesting issues when clients and servers don't share the same timezone.

A few gems from beer conversations

After the official sessions ended most of the attendees congregated to have diner followed by beers. Howard showcased his G1 phone running slapd while Ludovic was showing off an LDAP client application on his iPhone. And of course by then end of the conference both systems were connected: the iPhone was able to look up contact information on the G1 running slapd.

On an unrelated note OpenLDAP is faster than OpenDS, even in beer drinking. However the OpenLDAP project was compared to a Beetle car with a Porsche engine whereas OpenDS was actually building a Porsche.

Even though not all the players in the directory space were represented at the conference, most of the key players from the open source world were there presenting their work. Friendly competition exists amongst the different projects which turns into cooperation on topics that matters such as interoperability and data formats.

It seems that the directory world is rather small and its use cases are restricted to specific situations compared to RDBMS. This is rather unfortunate as directories offer a compelling alternatives to databases as a data store infrastructure. The community seems to be aware of this issue and is looking into breaking out of its traditional fields of applications.

Sep 11 - Sep 18 Wrap-up

Image-store-proxy

Package image-store-proxy to enable the Image Store tab in Eucalyptus. The package (python-image-store-proxy) has made its way to main and on the -server isos in time for alpha6 with the help of Thierry and Kees.

Server-karmic-directory-enabled-user-login

Kept on investigating the use of puppet to build an ldap/krb5 infrastructure on EC2. Integrated dnsmasq and puppetmaster configuration. Discovered a few bugs along the way and reported them upstream. My current work is available from an LP branch. And puppet is awesome!

Alpha6 ISO testing

Loads of alpha6 testing.

Landscape-client Stable Release Update

Reviewed the landscape-client and smart SRU requests from the Landscape team.

Bug scripts

With the help of Brian my bug scripts are now regularly run on qa.ubuntu.com. All bug lists used in the SRU review and the triaging process can be found on qa.ubuntu.com.

Misc

Updated my status report script to publish a draft of my activity report on my blog as the weekly "wrap-up".

Sep 07 - Sep 11 Wrap-up

Server-karmic-directory-enabled-user-login

Upload new sssd package to fix lintian errors and pull two fixes from upstream. Brainstorm with upstream about testing the package.

Prepare and upload openldap 2.4.18 to Karmic once the FFe was granted. That should complete the last part of the specification and brings disconnected mode support on the client via the cache overlay.

Looked into using puppet to build an openldap/krb infrastructure to test all the directory related components on the client side (sssd, openldap pcache overlay). The idea is to be able to pull up and down complete environments within minutes using a combination of EC2/UEC and puppet.

Follow up on puppet promotion into main for karmic.

Ended up writing a custom puppet type to handle slapd modules using the default karmic configuration. This gave me a good overview of how puppet is working.

Imate-store packaging

Looked at packaging. Follow-up call with Gustavo. Should have a package ready on Monday in time for alpha6. More polishing will be done for beta.

Apport in the default server install

Add apport to the default server install as requested by steve beattie for the karmic-qa-apport-in-ubuntu-server specification.

Linux-virtual missing virtio modules

Chase down and confirmed that linux-virtual kernel doesn't have any of the virtio modules. Bug 423426 is milestoned and should be on the release team radar. This has a high importance as virtio vms cannot boot in karmic. Tim is working on it.

Mysql maintenance

Caught up on (lots of) mysql 5.0 and 5.1 bugs. Updated DebuggingMysql page in the process of triaging bugs.

Upload mysql 5.0 and 5.1 to fix a couple of bugs. Both mysql-server-core-5.{0,1} packages provide mysql-server-core which should be used by packages requiring the mysqld binary (such as akonadi).

Sru-workflow

Write up a script to get a list of ubuntu-server SRU bugs assigned to people. This produces the remaining list to be reviewed during the team meeting with the updated SRU workflow in the ubuntu-server team.

Sponsoring

Reviewed checkbox merge proposal from Marc. Asked for a FFe as there is one new feature.

LDAPcon/LinuxCon

Arrange travel for LDAPcon/linuxcon in Portland, OR next week.