Trustedfirmware.org Project Migration Information Update

[joannafarley-arm]

As mentioned in my original Trustedfirmware.org Project Migration Information post (#655) I promised updates concerning the migration to hosting this project on https://www.trustedfirmware.org. We are planning and working towards a phased approach to the migration:

1st Phase: Migrate Repository, Patch Review, Wiki, Issues

• Existing Github project capabilities available on https://www.trustedfirmware.org these are still under construction.
• Repository: https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/
• Gerrit review system: https://review.trustedfirmware.org/
• Wiki: https://developer.trustedfirmware.org/w/tf_a/
• Issues: https://developer.trustedfirmware.org/project/view/1/
• Will have the same level of automated testing as we have today on Github where the Continuous Integration (CI) is run on Arm servers to build and test patches submitted for review. However because these servers are internal to Arm full details of CI results will not be externally visible and any result failures are communicated to partners via the patch review mechanism.
• The delivery for Phase 1 is targeted at the end of 1st quarter 2019. From April 2019 Github project requests (Pull Requests, Issues, etc) will be redirected to Trustedfirmware.org equivalents and the Github repository will become a mirror of the Trustedfirmware.org repository.

2nd Phase: Migrate Continuous Integration (CI) Build Capability

• Provide an additional capability where each patch submitted is built on TrustedFirmware.org servers and have the build results made available publicly.
• The goal is to make this CI publicly available in a new CI repository.
• The delivery for Phase 2 is targeted at the end of 2nd quarter 2019.

3rd Phase: Migrate CI Fixed Virtual Platforms (FVP) Test Capability

• Provide an additional capability to have each patch submitted tested on TrustedFirmware.org servers on a selection of Arm FVPs (https://developer.arm.com/products/system-design/fixed-virtual-platforms) and have the results made available publicly.
• Tests will include TF-A Tests (https://git.trustedfirmware.org/TF-A/tf-a-tests.git/about/) as well as end-to-end boot tests up to the Linux shell prompt.
• The goal is to make this CI publicly available in a new CI repository.
• The delivery for Phase 3 is targeted at the end of 3rd quarter 2019.

4th Phase: Migrate CI Hardware Test Capability

• Provide an additional capability to have each patch submitted tested on a LAVA physical board farm supported by TrustedFirmware.org servers initially equipped with Arm Juno boards
  (https://developer.arm.com/products/system-design/development-boards/juno-development-board) and have the results made available publicly.
• Tests will include TF-A Tests (https://git.trustedfirmware.org/TF-A/tf-a-tests.git/about/) as well as end-to-end boot tests up to the Linux shell prompt.
• The goal is to make this CI publicly available in a new CI repository.
• The delivery for Phase 4 is targeted at the end of 4th quarter 2019.

Any questions please let us at Arm know and we will try to clarify.

[DamianVesk]

end-to-end boot tests up to the Linux shell prompt

@joannafarley-arm Will there be an early "preview" of these tests available?
Thank you

[sandrine-bailleux-arm]

Hi @DamianVesk ,

I am unsure what you mean by having "an early preview" of these tests. Are you asking whether some of the end-to-end boot tests results will be visible before the end of the 3rd phase? At this stage, I'm afraid I don't have enough visibility to give a definitive answer.

I can certainly describe what these end-to-end boot tests consist in today, though, if that's what you are interested in.

The scripts that drive these boot tests first download various binary dependencies to assemble a complete software stack. Typically U-boot, a Linux kernel image, a RAMdisk or an OpenEmbedded root filesystem. All of these are coming from the Linaro release advertised in the TF-A User Guide.

Then we deploy this software stack on the target. When testing on a Fixed Virtual Platform (FVP), this is just a matter of pointing it to the location of the images on the host machine. For testing on real hardware (i.e. the Arm Juno development platform in our case) we rely on LAVA to abstract the flashing of the images and other details of interfacing with the board.

Then we start booting this software stack. Different pieces of software print messages on different UARTs. We monitor their output using some expect scripts. We look for specific string patterns like BL1: Booting BL2 or Linux version x.y to track the progress, until we see the Linux shell prompt.

If the expect scripts see all the strings we were expecting then the boot test is a pass. Otherwise, if some specific string never arrives, the scripts time out and the boot test is a fail.

[DamianVesk]

Hi @sandrine-bailleux-arm ,

Are you asking whether some of the end-to-end boot tests results will be visible before the end of the 3rd phase?

Yes, that was the intention of my question exactly. Thank you for the detailed answer, it's a very good description of what to expect from the tests in the future. I'll keep an eye for updates, if there's a mailing list I can subscribe for the development of this feature please let me know.

P.S. Great work on the TFTF test suite development! The tests are very insightful when it comes to dealing with the TF-A on the base FVP.

[sandrine-bailleux-arm]

Hi @DamianVesk ,

Thank you for the detailed answer, it's a very good description of what to expect from the tests in the future.

You're welcome. By the way, do note that we already do this level of testing today. It's just that the details and tests results are unfortunately not visible externally. I thought I would clarify, because you mentioned "expect from the tests in the future". The 3rd phase of this migration work is about moving this existing test infrastructure from Arm internal servers to trustedfirmware.org public ones, which in turn will give visibility to everyone.

I'll keep an eye for updates, if there's a mailing list I can subscribe for the development of this feature please let me know.

There is no mailing list dedicated to the trustedfirmware.org migration work but we do have TF-A and TF-A Tests mailing lists, see https://www.trustedfirmware.org/contact/.

P.S. Great work on the TFTF test suite development! The tests are very insightful when it comes to dealing with the TF-A on the base FVP.

I am very glad to know that is useful to you. The TF-A Tests project has already migrated to trustedfirmware.org so if you want to contribute, feel free to submit patches to review.trustedfirmware.org already. The contribution guidelines are outlined here.

[joannafarley-arm]

An update on the 1st Phase: Migrate Repository, Patch Review, Wiki, Issues migration.

This is expected to occur next week as per the timeline.

[joannafarley-arm]

A 2nd update on the 1st Phase: Migrate Repository, Patch Review, Wiki, Issues migration.

• The Repository migration has occurred today. The Github repository will be maintained as a readonly clone going forward.

• Patch Reviewing is open please refer to the updated contributing guidelines. Existing open Github PR's are being asked to migrate their requests. Any new Github PR's will be asked to migrate. Eventually an automated response will be put in place.

• All Github wiki content has been migrated with most being incorporated in the repository docs directory. A smaller wiki is maintained on the new site. The existing Github Wiki will eventually be redirected.

• Existing issues will eventually be closed or migrated to the new issue/task tracking system. Eventually new issues's will be disabled and an automated response will be put in place.

• A landing page on the new site is the TrustedFirmware.org TF-A Dashboard that will provide access to all the new sites capabilities. You can continue to authenticate on this new site with your Github credentials. Once authenticated you should become a member of the Trusted Firmware A project.