Wednesday, June 18, 2008

Example Headless build for a RCP product with p2

*UPDATE* For best results, I would suggest using the upcoming 3.4.1 release. Candidate builds are available from the download page under "3.4.1 Stream Builds", RC3 is available now.

I have prepared an example build setup that builds the RCP Mail Template and produces a p2-ized product. All the files needed for this are provided HERE.

The first step is to set up Eclipse with the RCP Delta pack which is required for the headless build of RCP products:
  1. Get the Eclipse SDK and the RCP Delta pack and unzip into the same location on disk.
  2. Start Eclipse and go to Window -> Preferences -> Plug-in Development -> Target Platform.
  3. Uncheck the option "Build target platform based on the target's installed plug-ins."
Create a new plug-in "com.acme.rcp" based on the RCP Mail Template. Create a new Product Configuration file "acme.product" for this plug-in. On the overview tab of the product editor, I set the id "com.acme.rcp.product" and the application "com.acme.rcp.application". I also set the version to be "1.0.0.qualifier".

The p2 Installable Unit (IU) that will be created for this product will take its id and version from the product id and version set here. I am setting the version to end in "qualifier" so that I can replace the qualifier at build time with the build id.

The product configuration for this example is based on plug-ins. On the configuration tab remove all plug-ins, add the plug-ins listed below and click Add Required Plug-ins.
  • com.acme.rcp
  • org.eclipse.equinox.p2.exemplarysetup
  • org.eclipse.equinox.p2.ui.sdk
  • org.eclipse.equinox.p2.touchpoint.eclipse
  • org.eclipse.equinox.p2.touchpoint.natives
  • org.eclipse.ecf.filetransfer
  • org.eclipse.ecf.provider.filetransfer
The product can now be run from overview tab of the product editor. If you do so you will notice the Help -> Software Updates... menu item. Selecting Software Updates at this time gets you a "This installation has not been configured properly for Software Updates" message.

Setting up the headless build

I like to run the headless build from inside Eclipse using a launch configuration. To do this, first import (Import... -> Plug-in Development -> Plug-ins and Fragments) "org.eclipse.pde.build" from the target platform into your workspace as a binary project.

Next, create a new general project named "Builder" and copy
org.eclipse.pde.build/templates/headless-build/build.properties"
/customTargets.xml"
into the builder project.

I want my headless build to do 3 things differently from a normal build:
  1. Set the version number in the product file to match the build id.
  2. Generate p2 metadata
  3. Use the generated metadata to perform p2 installs to get the final archives

1: Set the version number in the product file.

See the Build/customTargets.xml/preSetup target. We copy the "acme.product" file out of the com.acme.rcp bundle and into our buildDirectory. We then use the ant replace task to replace the version qualifier based on the time stamp. This uses a "acmePlugin" property which we will define separately. We also save the time stamp to a properties file that we read later when calling the p2 director.

2: Generate p2 metadata

PDE/Build provides integration with p2 to automatically generate p2 metadata. See Builder/build.properties where the following properties have been set:
    generate.p2.metadata = true
p2.metadata.repo=file:${buildDirectory}/repo
p2.artifact.repo=file:${buildDirectory}/repo
p2.flavor=tooling
p2.publish.artifacts=true

3: Perform p2 installs and archive the results

We do this in the Builder/customTargets.xml/postBuild target. We exec a new instance of eclipse to run the director application to install the product into a temporary directory and then we zip up the results.

We are running the same eclipse install again to do this. This means that any repositories that the eclipse install is set up with will also be considered during the director call (in addition to the repositories we pass on the command line).

The rest: build.properties

There are a few other properties to set in the build.properties file. We also set a number of properties on the command line in the launch configuration.

In the build.properties we set the product property to be the one that has the modified version qualifier. We also set the configuration we are building for.
 product=${buildDirectory}/acme.product
configs=win32,win32,x86
On the command line we are setting:
  • pluginPath: allows us to not copy the com.acme.rcp plugin into the buildDirectory.
  • buildDirectory: where all scripts will be generated and the location of our modified acme.product file.
  • baseLocation: We use the eclipse install we are running as our base.
  • acmePlugin: Define the location of the acme plugin for our custom task.
All these properties are set in the provided "Build Acme Product.launch" launch configuration and values are set at launch time using variables.

Results

Go to the Run Configurations... dialog and run the "Build Acme Product" launch config. The resulting zip should be under Builder/result. The project may need to be refreshed to see the results.

Running the build multiple times will result in the repository containing multiple versions of the product. You can then use that repository to update the product from one version to the next.

Problems Encountered

I did encounter some problems while creating this example.
  • Bug 237662: The metadata generator currently does not properly handle the new p2 simpleconfigurator style of config.ini and loses some start level information. This causes the installed product to not start. Workaround for this example is to not include org.eclipse.equinox.simpleconfigurator in the product. I will provide another example later to work around this problem.

  • Bug 222969: When including the org.eclipse.equinox.simpleconfigurator bundle in the product, the install is getting a bad relative path in the config.ini. Since we are excluding the simpleconfigurator because of the previous bug, this doesn't affect our example.

  • Bug 237647: NullPointerException in the director. This only occured once, workaround was simply to remove the bad profile directory under my eclipse install (eclipse/p2/org.eclipse.equinox.p2.engine/profileRegistry/ACMEProfile.profile)

25 comments:

Chris Aniszczyk (zx) said...

Thanks Andrew, I see those commit rights to PDE Doc has helped a bit ;)

Andrew Niefer said...

I wasn't quite sure where to put mini articles/tutorials like this. But I've just noticed we have a pde-build-home project in cvs that shows up as http://www.eclipse.org/pde/pde-build
That page probably deserves a facelift.

Chris Aniszczyk (zx) said...

I have a wiki setup for PDE

http://wiki.eclipse.org/PDE

The PDE Build section is empty ;o!

Richard C. Gronback said...

Thanks, this was helpful. I only wish I had paid closer attention to the "don't include the simpleconfigurator" bundle tip at the bottom ;)

BTW, there's a typo in customTargets.xml (%{buildDirectory} > ${buildDirectory}). I made additional changes to build on the Mac, but then didn't find an easy way to account for multiple target platform builds. I'd have expected I could have used a customAssembly.xml for this, but no luck. Do you have a suggestion for this, and are there plans to make the p2-ification of product builds easier in the future? (e.g. adding/generating a set of director Ant tasks for accomplishing this in a more "native" way).

Thanks again,
Rich

Andrew Niefer said...

Richard,
I fixed the typo and updated the zip on the bug.

Unfortunately, the p2 metadata generation happens too late in the process to take advantage of the assembly call backs. So add platforms must be done by adding more calls in the post build.

I will be reviewing all of this for 3.5. The current integration is really just about generating metadata, we want to bring this forward and have the actual install done automatically as well.

Igor said...

Hello Andrew,

First of all thanks a lot for the articles. But I still have some problems running it. During the run.director target im getting following nullpointerexception:
run.director:
[exec] Installing com.acme.rcp.product 1.0.0.v200808281244.
[exec] Operation completed in 20656 ms.
[exec] java.lang.NullPointerException
[exec] at org.eclipse.osgi.framework.internal.core.AbstractBundle.getBundleDescription(AbstractBundle.java:1337)
[exec] at org.eclipse.osgi.framework.internal.core.AbstractBundle.getResolutionFailureMessage(AbstractBundle.java:1376)
[exec] at org.eclipse.osgi.framework.internal.core.BundleHost.startWorker(BundleHost.java:305)
[exec] at org.eclipse.osgi.framework.internal.core.AbstractBundle.start(AbstractBundle.java:265)
[exec] at org.eclipse.osgi.framework.util.SecureAction.start(SecureAction.java:400)
[exec] at org.eclipse.core.runtime.internal.adaptor.EclipseLazyStarter.postFindLocalClass(EclipseLazyStarter.java:111)
[exec] at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLocalClass(ClasspathManager.java:427)
[exec] at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.findLocalClass(DefaultClassLoader.java:193)
[exec] at org.eclipse.osgi.framework.internal.core.BundleLoader.findLocalClass(BundleLoader.java:368)
[exec] at org.eclipse.osgi.framework.internal.core.SingleSourcePackage.loadClass(SingleSourcePackage.java:33)
[exec] at org.eclipse.osgi.framework.internal.core.BundleLoader.findClassInternal(BundleLoader.java:432)
[exec] at org.eclipse.osgi.framework.internal.core.BundleLoader.findClass(BundleLoader.java:397)
[exec] at org.eclipse.osgi.framework.internal.core.BundleLoader.findClass(BundleLoader.java:385)
[exec] at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadClass(DefaultClassLoader.java:87)
[exec] at java.lang.ClassLoader.loadClass(Unknown Source)
[exec] at java.lang.ClassLoader.loadClassInternal(Unknown Source)
[exec] at org.eclipse.equinox.internal.p2.engine.MetadataCache$1.notify(MetadataCache.java:81)
[exec] at org.eclipse.equinox.internal.p2.core.ProvisioningEventBus.dispatchEvent(ProvisioningEventBus.java:86)
[exec] at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:211)
[exec] at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:321)

And there is no ACMEProfile.profile, so I cant delete it. Despite this exception the build is always successful and I can find the App under results. But after launching it, Im getting following Exception in the log file:
java.lang.IllegalStateException: Unable to acquire application service. Ensure that the org.eclipse.core.runtime bundle is resolved and started (see config.ini).
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:74)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:382)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:549)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:504)
at org.eclipse.equinox.launcher.Main.run(Main.java:1236)

Do You have any idea, what went wrong? Thank you for your help.

Igor.

Andrew Niefer said...

Sorry Igor, I haven't seen that problem before. You should open a bug on the Equinox Framework with that stack trace and any other details.

plizak said...

Found your online home...

Peter

msacarny said...

Great article!
Newbie question: When I build the example, the resulting zip has no eclipse.exe or custom launcher. As intended?

If this is not standalone, then what is the recommended install technique? Copy result over existing clean RCP Eclipse?

Thanks!

msacarny said...

I've got this working as a plug in. Thanks. Any directions on the Feature version?

Andrew Niefer said...

The example is expected to have a launcher in the result and be runnable. Do you have the delta-pack installed? The executable is expected to come from org.eclipse.equinox.executable feature which is in the delta-pack.

> Any directions on the Feature version?
I'm not clear on what you are asking here.

msacarny said...

Your example is for a Plugin based product, which I got working. I'm trying to extend the example to a Feature-based product, so I can include additional Features and plugins. I'm trying to puzzle out how to reconcile this with the org.eclipse.pde.build.container.feature scripts.

Kai Tödter said...

Hi Andrew,

thanks for the article. I followed your instructions and getting errors during the director run (Windows Vista, Java 6 update 10, Eclipse 3.4 + Delta Pack). I only displayed the first of a long row of unsatisfied dependencies:

run.director:
[exec] Installing com.acme.rcp.product 1.0.0.v200809170951.
...
[exec]
[exec] !ENTRY org.eclipse.equinox.p2.director 2 0 2008-09-17 09:56:00.072
[exec] !MESSAGE Problems resolving provisioning plan.
[exec] !SUBENTRY 1 org.eclipse.equinox.p2.director 2 0 2008-09-17 09:56:00.073
[exec] !MESSAGE Unable to satisfy dependency from org.eclipse.equinox.p2.ui 1.0.0.v20080530-1237 to requiredCapability: osgi.bundle/org.eclipse.equinox.simpleconfigurator/0.0.0.
....
[exec] Result: 13

Any hints?

Thanks

Kai

Andrew Niefer said...

Kai, I'm seeing this as well. I don't really have time to figure why this was working previously but not now.

I would suggest using a 3.4.1 build (RC3 is here) and including the simpleconfigurator in the plugins list.

msacarny, you should be able to just change the .product file to be based on features. I would also suggest using 3.4.1 where some of the bugs mentioned in the post are fixed.

msacarny said...

Andrew,
I've made progress on a Feature-based product using 3.4.1RC3.
* I added a Feature com.acme.rcp.feature with ID com.acme.rcp and branding com.acme.rcp. This Feature has plugin com.acme.rcp.
* I changed acme.product configuration to be feature-based and added the following features:
- com.acme.rcp
- org.eclipse.rcp
- org.eclipse.equinox.p2.user.ui
- org.eclipse.platform
* I changed the launch app to org.eclipse.ui.ide.workbench.
* I changed the Build acme launch config so that:
-DpluginPath=${resource_loc:/com.acme.rcp};${resource_loc:/com.acme.rcp.feature}

When I do this, the product builds and runs! However, the Feature Details button is missing from Help, About. There is no Features folder in the zipped product.

If I launch from the workbench (acme.product, Overview, Testing, Launch) then Feature Details is present, but com.acme.rcp is not in the list of features.

Your example has been so helpful. Any idea what may be wrong here?
Thanks, Mike

Richard C. Gronback said...

Try adding this argument: -profileProperties org.eclipse.update.install.features=true

msacarny said...

Adding -profileProperties to customTargets.xml after -bundlepool seems to do the trick. Thanks!

Now I just need to figure out how to get the Feature Copyright and License info to show in Help, Updates, [product], Properties. The data in feature.xml does not display for my Product.

Richard C. Gronback said...

Do you have an associated branding plug-in for your feature?

msacarny said...

Andrew,
The RCP Plug in headless build does not generate a "dropins" folder during the build. How should this be created?

Thanks,Mike

msacarny said...

Richard,
Yes, I have an associated branding plugin for the feature. This has some "about" text but I don't know where to put the license\copyright info.

When I display Feature Details, my Feature is in the list but when selected the lower dialog says "No further information is available for this feature." The Feature.xml, Information tabs have license and copyright stuff, however, it just doesn't display.

Regards,Mike

Richard C. Gronback said...

Mike,

Well, there are a lot of little pieces here. Perhaps the best thing to do is take a look at the Amalgam project's 'DSL Toolkit' product/feature I've been working on at dev.eclipse.org/cvsroot/modeling/org.eclipse.amalgam/

The org.eclipse.amalgam.dsltk-feature project includes a feature.properties and license.html file (don't forget to add everything to build.properties), with a corresponding org.eclipse.amalgam.dsltk branding plugin, which also is where the product contribution is synchronized with the .product definition (found in the *.releng project). Between these two projects, you should find the answers.

Best,
Rich

Andrew Niefer said...

Thanks for your answer Richard. Mike, a better forum for your questions would be the newsgroups: eclipse.platform.rcp or eclipse.platform.pde.

The simplest way to get an empty dropins folder in this example would be simply create one in the customTargets.xml/postBuild step before zipping up the result.
You could also provide one using the rootfiles properties

msacarny said...

Andrew,
I've built the p2-ized headless acme example and all seems well until I try and dropin plugins into the target build. That is, I unzip the acme-p2-RCP-win32-...zip and drop a plugin into the "plugins" folder (there being no "dropins" folder). The plugin does not activate or appear in the Help, About list of plugins.

I used an Exported plugin of the "Hello World" project template as a sample plugin.

If I make a Feature of the Hello plugin and Export it with metadata, then I can include it successfully via Software Updates, Available Software, etc. but shouldn't dropping into the plugins folder "just work"?

Thanks, Mike

msacarny said...

Andrew,
Our last posts crossed. I'll post in the newsgroups recommended from now on. Thanks again Andrew and Richard, Mike.

Kai Tödter said...

Andrew, thanks a lot again! I just adopted your scripts to my mp3 manager project. The 3.4.1 headless p2 build works very well :)

Best wishes,

Kai