Thursday, March 26, 2009

Building p2 RCP products in Eclipse 3.5M6

We can also call this "Building for the Cloud" just to take advantage of the newest buzz word.

Susan Franklin McCourt put together a nice wiki page covering how to add p2 self-updating support to your RCP application. One of those examples was called "Updating from the Cloud".

As part of preparing for EclipseCon, I put together a simple releng product build that builds this cloud product for p2. The cloud bundle "org.eclipse.equinox.p2.examples.rcp.cloud" is available in CVS together with a releng project that I created.

[Edit 2013/07/03: Eclipse CVS has migrated to git starting in 2010.  The examples are now available under the examples folder in http://git.eclipse.org/gitroot/pde/eclipse.pde.build.git. The example has not been updated to work with git and has an obsolete CVS entry in the .map file.]

To run this example, check both projects out into your workspace, and read the rcp.cloud.releng/readme.txt.

Builder Setup

The releng project serves at the builder for a headless build. The build.properties file is a copy of the template provided by PDE/Build with some things we don't need removed. We modified the following properties:
  • product : This defines the .product file which specifies how to create the cloud product. The first segment of this path is the plug-in id for the cloud project that contains the file.
  • configs : Here we define the platforms we are building for. This is currently set to build windows, other platforms can be added. (The default here is *,*,* which is platform independent pieces, which doesn't make sense for a product).
  • J2SE-1.5 : The cloud project has a Bundle-RequiredExecutionEnvironment of 1.5. We set this property to be the bootclasspath to use for compiling 1.5. See the eclipse help for more details on these properties. We also set CDC-1.1/Foundation-1.1 because we had the org.eclipse.osgi bundle in our workspace (and so were compiling it for the product) and osgi requires that EE defined in order to compile properly.
  • p2.gathering : We set this to true to use the new support for publishing metadata directly from source. This will also automatically install the product using the p2 director. More details on this below.
  • p2.metadata.repo, p2.artifact.repo : In addition to the installed product, we also want a p2 repository.

We also provide a simple ant script named "buildProduct.xml". This is a very simple script that does the following:
  1. Set the baseLocation. This is the location of your target binaries against which you want to compile. In the script we set it to ${eclipse.home} which is a property that is automatically set to be the eclipse that is running the build.
  2. Find the delta pack. The delta pack is required for headless product builds because it contains all the platform specific fragments, and org.eclipse.equinox.executable feature which contains launchers for all the platforms. Download the 3.5M6 deltapack from here. The deltapack location will be added to the pluginPath property.
  3. builder : the location of the build configuration files (ie: build.properties). We set it to be the directory containing the buildProduct.xml script.
  4. buildDirectory: the directory where everything happens, set it to a subfolder of our builder.
  5. pluginPath: the location to find more plugins and features. Here we add the workspace (one level up from the builder), and the deltapack.
  6. Run the build. The property ${eclipse.pdebuild.scripts} is automatically set to the location of the PDE/Build scripts directory. Here we call the product build.
  7. Victory! : copy the resulting archive into the root of the releng product.
The results of this build is a fully provisioned, p2 enabled product with an accompanying repository.


p2.gathering

In 3.4, p2 metadata was generated by running the metadata generator on the binary jars that were the result of the build. At the end of the build you were left with a repository and you need to perform a director install for get a fully p2-enabled product.

In 3.5 we have this new property p2.gathering. Setting this to true does a couple of things:
  • Generate metadata from source. PDE/Build extends the p2 publisher to publish metadata and artifacts directly from source (and the compiled .class files) into a repository. There are no intermediate steps. (This also has the side effect of increased performance).
  • Feature Builds result in a repository that is a group of all the platforms that were built. (This is an implicit groupConfigurations=true).
  • Product Builds result in a fully installed p2 enabled product. This removes the need for a manual director call after the build is complete. The product build can also create a repository containing the IUs required to install the product.
The 3.4 properties still work for doing the old style generation.

12 comments:

Victor said...

Remarks about p2.inf

p2.inf contained in example projects don't work for me in 3.5 M6.

I spent some time and I have find working variant:

instructions.configure=addRepository(location:http${#58}//www.stepanych.th/update/,type:0,name:TIMMY,enabled:true);addRepository(location:http${#58}//www.stepanych.th/update/,type:1,name:TIMMY,enabled:true);

Andrew Niefer said...

Victor, I haven't tried re-running the example, but I don't see any change in your version that should make a difference.

The order of the parameters does not matter. Both 'name' and 'enabled' are optional. (Default value of enabled is true).

Victor said...

Yes, 'name' is optional. I find this parameter when looking through sources. But no comments. Only facts. :(

JesperE said...

I tried building according to the instructions, using the Build Cloud Example.launch, and Eclipse SDK 3.5RC3, but get the following errors. What am I doing wrong?

[echo] !ENTRY org.eclipse.equinox.p2.director 4 1 2009-06-12 12:44:21.641
[echo] !MESSAGE Cannot complete the install because one or more required items could not be found.
[echo] !SUBENTRY 1 org.eclipse.equinox.p2.director 4 0 2009-06-12 12:44:21.641
[echo] !MESSAGE Software being installed: RCP Mail Example (Cloud Update) 1.0.0.200906121243 (org.eclipse.equinox.p2.examples.rcp.cloud.product 1.0.0.200906121243)
[echo] !SUBENTRY 1 org.eclipse.equinox.p2.director 4 0 2009-06-12 12:44:21.641
[echo] !MESSAGE Missing requirement: Equinox Provisioning Update Checker 1.1.0.v20090520-1905 (org.eclipse.equinox.p2.updatechecker 1.1.0.v20090520-1905) requires 'package org.eclipse.equinox.internal.provisional.p2.repository 0.0.0' but it could not be found
[echo] !SUBENTRY 1 org.eclipse.equinox.p2.director 4 1 2009-06-12 12:44:21.641
[echo] !MESSAGE Cannot satisfy dependency:
[echo] !SUBENTRY 2 org.eclipse.equinox.p2.director 4 0 2009-06-12 12:44:21.641
[echo] !MESSAGE From: RCP Mail Example (Cloud Update) 1.0.0.200906121243 (org.eclipse.equinox.p2.examples.rcp.cloud.product 1.0.0.200906121243)
[echo] !SUBENTRY 2 org.eclipse.equinox.p2.director 4 0 2009-06-12 12:44:21.641
[echo] !MESSAGE To: org.eclipse.equinox.p2.updatechecker [1.1.0.v20090520-1905]

Andrew Niefer said...

JesperE, A new bundle "org.eclipse.equinox.p2.repository" was added to p2 in 3.5M7. The cloud.product file in the example was out of date. On the dependencies tab of the product editor, click "Add Required Plug-ins" to get the repository bundle added to the product.

I have updated the version in CVS.

JesperE said...

Thanks, great. It works now.

Daniel S said...

Hello,

I'm trying this example to run http://wiki.eclipse.org/Equinox/p2/Adding_Self-Update_to_an_RCP_Application#Headless_Updating_on_Startup

and it's running, but update is not working..I'm getting following errors

osgi> LOAD REPOS
GET UPDATE LIST
2
[org.eclipse.equinox.p2.examples.rcp.prestartupdate.product 1.0.0.200908041846,
org.eclipse.equinox.p2.examples.rcp.prestartupdate 1.0.0.200908041846]
[org.eclipse.equinox.p2.examples.rcp.prestartupdate.product 1.0.0.200908041848,
org.eclipse.equinox.p2.examples.rcp.prestartupdate 1.0.0.200908041848]
Status OK: org.eclipse.core.runtime code=0 OK null
Status ERROR: org.eclipse.equinox.p2.engine code=4 An error occurred while colle
cting items to be installed null children=[Status ERROR: org.eclipse.equinox.p2.
engine code=0 session context was:(profile=profile, phase=org.eclipse.equinox.in
ternal.provisional.p2.engine.phases.Collect, operand=, action=). null Status ERR
OR: org.eclipse.equinox.p2.artifact.repository code=0 No repository found contai
ning: osgi.bundle,org.eclipse.equinox.p2.examples.rcp.prestartupdate,1.0.0.20090
8041848 null Status ERROR: org.eclipse.equinox.p2.artifact.repository code=0 No
repository found containing: binary,org.eclipse.equinox.p2.examples.rcp.prestart
update.product_root.win32.win32.x86,1.0.0.200908041848 null]

As you can see, there is an Update, but its not downloading...
I'm using Apache @ localhost:5000 , I heard about an issue with a port rather then 80, but it should be fixed by now ..
My Repository is looking like this:

repo
-- binary
-- plugins
-- artifact.xml
-- content.xml

Thank you

Andrew Niefer said...

I expect the problem is related to the server being on port 5000 but I don't know much about the transport layer here, I don't have any suggestions for you. I assume your repository URLs are something like http://localhost:5000/repo.

Ralf Ebert said...

Hi Andrew,

thanks a lot for the example code, it's extremely cool to build this from the spot on my win32 VM!

I get some warnings though (I highlighted them as three different sections in the document):
http://pastie.org/pastes/581949

I'm not sure if I can ignore them, from the first look I cannot see any obvious reasons for them. Do you know right away what's wrong or shall I investigate this?

I also tried this on OS X, building for "macosx, cocoa, x86". The build is successful, but I get error messages about "java.lang.NoClassDefFoundError: org/eclipse/ui/plugin/AbstractUIPlugin" in the compile log and in the configuration log file when I start the built application...

Build log:
http://pastie.org/581959

Any ideas about this?

Greetings,
Ralf

Andrew Niefer said...

Ralph, none of these warnings are things that you need to worry about.

1) [eclipse.generateFeature] Some inter-plug-in dependencies have not been satisfied.
This just means that there were some bundles in the target that did not resolve for some reason. This is only a problem if you are actually trying to include one of those bundles in your product. Most common is the "Another singleton version selected". When there is more than one version of a singleton bundle, only one will be able to resolve. The JavaSE-1.6 warnings might be because only a Java 1.5 boot classpath was defined in the build.properties.

2) Discouraged Access warnings.
This is because p2 has not yet declared any official API. Everything is provisional and is marked internal. The warnings help remind you that the API might change before it is official.

3) [p2.mirror] Unable to satisfy dependency from org.eclipse.equinox.executable.feature.group to org.eclipse.equinox.launcher.gtk.solaris.sparc
This would be because you only ran the build for a few platforms, so there won't be metadata for some of the launcher fragments. The executable feature is only mirrored here because of a quirk of pde.build and won't actually be required by your product. Your product will include branded versions instead for those platforms you built for.

4) "java.lang.NoClassDefFoundError: org/eclipse/ui/plugin/AbstractUIPlugin" exceptions
This is probably bug 285888. This won't actually hurt your install because the director is a headless application.

Gunnar said...

Andrew, I have a feature-product build running which produces a nice ZIP containing the branded launcher and the produced stuff.

When I enable p2.gathering the resulting zip file suddenly also contains a "binary" folder with the launcher artifacts. It looks like the resulting zip now also is a p2 repository.

Is it possible to produce a zip file which really is just the installed product and not a p2 repository itself?

For example, what I'd like to produce in my builds is:

1. One zip file containing features, plug-ins, fragments and launchers for all supported platforms

2. One zip per platform that does not contain things like a "binary" folder


Does PDE Build supports this out of the box or do I have to tweak the build files?

Andrew Niefer said...

Gunnar,
A product build with p2.gathering=true should produce for each configuration an archive which is the result of calling the director to install the product on that platform. You would not expect to see the binary folder in these installs (although there may be a eclipse\p2\org.eclipse.equinox.p2.core\cache/binary folder).

If you set the p2.metadata.repo and p2.artifact.repo properties, build will mirror to that location all the metadata and artifacts for the product. You would need to zip up that repo yourself.

In contrast, a feature build will produce a single p2 repo for all platforms (comparable to setting groupConfiguration & outputUpdateJars in older builds).

A build is considered a product build if you define the product property, and a feature build otherwise.