Dirac v6r11

The main point of this version is to remove the uselessly complex ReplicaManager class. It does not contain any change from the logic perspective, but introduces slightly different interface. It is a first step in the implementation of RFC #17. The replacement of PFN with (LFN,SE name) is to happen in v6r12.

ReplicaManager class deprecated

The calls to the ReplicaManager class have to be replaced by calls :

• to the FileCatalog, if the method acts only on the FileCatalog. It is the case for all methods of the ReplicaManager which contains "Catalog" (eg. getCatalogReplicaStatus, removeCatalogReplica, etc).
• to the StorageElement if the method acts only on the StorageElement. It is the case for all methods of the ReplicaManager which contains "Storage" (eg. getStorageFileMetadata, prestageStorageFile, etc).
• to the new DataManager class if it implies both the StorageElement and the FileCatalog.

The SingleFile and SingleDir parameters are deprecated as well. The enforced return type convention is the successful/failed dictionaries. However, a utility function is provided to mock the deprecated arguments :

res = ReplicaManager().doSomething(arg1, arg2, singleFile = True )

becomes

from DIRAC.Core.Utilities.ReturnValues import returnSingleResult

res = returnSingleResult(DataManager().doSomething(arg1, arg2 ))

The catalogs parameter of the ReplicaManager is also removed. It allowed you to specify on which catalogs your method should act. This information should be now given in the constructor of the DataManager or the FileCatalog itself, and all the methods called with these instances will act only on the given catalogs.

rm = ReplicaManager()
res = rm.registerFile( fileTuple, catalog = 'LFC' )

becomes

dm = DataManager(catalogs = ['LFC'])
res = dm.registerFile( fileTuple )

StorageElement class

The StorageElement class was slightly refactored in order to prepare the replacement of PFN with (LFN,SE name). The interface remains the same except for getPfnForProtocol and getPfnForLfn which now follow the successful/failed return convention, and a new getLfnForPfn method, that replaces the eponymous methods in the FileCatalog and the ReplicaManager.

Also, it is now encouraged to call the StorageElement methods with an LFN rather than a PFN. If you give it a PFN, a warning will be issued.

Few changes to ReqDB

PR https://github.com/DIRACGrid/DIRAC/pull/1815 fixes the missing FK of ReqDB. The following changes have to be applied by hand:

• PK of Request table is RequestID now (was RequestID + RequestName)
• RequestName is UNIQUE (unique index)
• added FK to Operation table to reference the request (index on Operation.RequestID automatically created)
• added FK to File table to reference the operation (index on File.OperationID automatically created)

Few changes to field definitions affecting several DBs/tables

Fields defined as VARCHAR(256) are now VARCHAR(255). This eventually fixes issues when utf8 encoding is used in the database and an index is created on these fields: it prevents mysql error 1071. Current databases in deployment to be changed by hand are:

• AccountingDB (several tables: keyFields, valueFields, bucketsLength)
• TransferDB (Channel: SourceSE, SourceSURL, TargetSE, TargetSURL; FTSReq: SourceSE, TargetSE)
• NotificationDB (ntf_AlarmLog: Modifications)
• JobDB (Jobs: ApplicationStatus)
• JobLoggingDB (LoggingInfo: ApplicationStatus)

Reports made to the Accounting

To answer the ticket https://github.com/DIRACGrid/DIRAC/issues/1331, the reported value when removing a file is now the data size deleted. There is also a report done when using getFile.

API changes

• Job.setSystemConfig() method is dropped. Use setPlatform() which has the same meaning but is consistent with the rest of the WMS nomenclature

WorkloadManagement

Pilot script installs DIRAC client with python 2.7 by default and with LCG bindings 2014-04-09

New PoolComputingElement class to be used to explore algorithms of using multi-core computing slots

Possibility to specify arbitrary Tags in the computing resources description that can be matched to the job requirements

Changes for ProxyDB [as from DIRAC v6r11p3]

As committed within https://github.com/DIRACGrid/DIRAC/pull/1988 the table ProxyDB_Log has a new primary key, as well as an index. These changes HAVE to be applied, by hand, BEFORE the release is installed.

ALTER TABLE `ProxyDB`.`ProxyDB_Log` 
 ADD COLUMN `ID` bigint NOT NULL AUTO_INCREMENT PRIMARY KEY FIRST,
 ADD INDEX `Timestamp` (`Timestamp`);

Timing can be problematic, with a possible high point of 1 minute per 1M rows (this table, up to now, has only grown).

#FrameworkSystem Change to the UserProfileDB

ALTER TABLE up_ProfilesData ADD PublishAccess varchar(10) DEFAULT 'USER';