org.forgerock / org.forgerock.opendj

29 Jul, 2014

1 commit

25 Jul, 2014

4 commits

  • 5ec3b0d6   OPENDJ-1023 OPENDJ-1024 (CR-4092) Provide duration and warm-up parameters to the xxxrate tools ... Browse Code » 
    Adding two new parameters to the xxxrate tools, maximum duration time and warm-up duration time.

** PerformanceRunner.java
* Adding a timer thread for the max duration feature and add warm-up implementation.
** AddRate.java SearchRate.java
* Adding resetStats method to reset local statistics (i.e extra columns)
** tools.properties
* Adding warm-up message
** AuthRateITCase.java
** Adding a test with warm-up phase
 

git-svn-id: https://svn.forgerock.org/opendj/trunk@10932 41b1ffd8-f28e-4786-ab96-9950f0a78031
    gaetan
     
  • c4e6860d   Fixed regressions introduced in r10929. ... Browse Code » 
    PBKDF2PasswordStorageSchemeTestCase.java: ADDED

git-svn-id: https://svn.forgerock.org/opendj/trunk@10931 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • c2c75bfb   OPENDJ-1441 (CR-4082) Persistent searches on external changelog do not return ch… ... Browse Code » 
    …anges for new replicas and new domains

Created ECLMultiDomainDBCursor to only let ECL enabled domains reach the ChangeNumberIndexer.

ChangeNumberIndexer.java:
Wrapped the MultiDomainDBCursor into an ECLMultiDomainDBCursor.

ECLMultiDomainDBCursor.java: ADDED

JEChangelogDB.java, FileChangelogDB.java:
Removed unnecessary calls to MultimasterREplication.isECLEnabledDomain().

git-svn-id: https://svn.forgerock.org/opendj/trunk@10930 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • eeb64667   Code cleanup for password storage schemes. ... Browse Code » 
    Used AutoRefactor.
Reduced variable scopes.

PBKDF2PasswordStorageScheme.java:
Extracted methods getDigestBytes(), encodeAndMatch(), 
Removed unnecessary use of StringBuilders.
In passwordMatches() and authPasswordMatches(), used String.indexOf().

PKCS5S2PasswordStorageScheme.java:
See general changes.

PKCS5S2PasswordStorageSchemeTestCase.java:
In getTestPasswords(), reused the super method and filtered entries.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10929 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

23 Jul, 2014

4 commits

  • d203eca1   OPENDJ-1441 (CR-4037) Persistent searches on external changelog do not return ch… ... Browse Code » 
    …anges for new replicas and new domains

Code Review: Nicolas Capponi

Fixed regression introduced in r10912. It made fractional replication tests fail.
ChangeNumberIndexer.initialize() must ignore ReplicaOfflineMsgs, just like ChangeNumberIndexer.run().


git-svn-id: https://svn.forgerock.org/opendj/trunk@10928 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • cfab9f24   OPENDJ-1074 (CR-4030) Implement combined add/del rate tool ... Browse Code » 
    Adding an add/del rate tool named addrate.
-opendj-core
    ** EntryGenerator.java:
    * Changing the subtemplate parsing behavior, add the "generated branches" options
    ** TemplateFile.java
    * Adding the "generated branches" options
    ** addrate.template
    * New template used to generate entries

-opendj-ldap-toolkit
    ** addrate (bat and bin)
    * Adding new tool scripts for unix and windows
    ** AddRate.java
    * addrate tool new class
    ** PerformanceRunner.java
    * Using parameter object in constructor, adding some messages localization
    ** PerformanceRunnerOptions.java
    * Helps to simplify the way to initialize a PerformanceRunner instance
    ** MakeLDIF.java
    * Little refactoring in order to allow the addrate tool to use some make-ldif parameters
    ** tools.properties
    * Adding addrate tool message and also some new generic messages
    ** Minor changes (constructor) in other tools and tests files

git-svn-id: https://svn.forgerock.org/opendj/trunk@10927 41b1ffd8-f28e-4786-ab96-9950f0a78031
    gaetan
     
  • c66d4c3c   OPENDJ-1441 (CR-4037) Persistent searches on external changelog do not return ch… ... Browse Code » 
    …anges for new replicas and new domains

Fix done in r10925 ported to JE changelog implementation
  

git-svn-id: https://svn.forgerock.org/opendj/trunk@10926 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     
  • e2727c92   OPENDJ-1441 (CR-4037) Persistent searches on external changelog do not return ch… ... Browse Code » 
    …anges for new replicas and new domains

Fixed a problem introduced in r10918.

In getExistingOrNewDomainMap(DN) method, the MultimasterReplication.isECLEnabledDomain() method is called.
However, this method blocks forever at replication startup because MultimasterReplication state is equals to STARTING
and is only set to RUNNING in code that occurs after in the same thread.

Code sequence is :
1. SynchronizationProviderConfigManager calls MultimasterReplication.initializeSynchronizationProvider()
   try to call MultimasterReplication.isECLEnabledDomain() but MultimasterReplication state is equals 
   to STARTING so it waits indefinitely on a state change
2. SynchronizationProviderConfigManager calls DirectoryServer.registerSynchronizationProvider(provider)
   MultimasterReplication state is set to RUNNING

Add a check on registeredMultiDomainCursors to ensure that isECLEnabledDomain() is not called when 
the replication server is starting (as no cursors can be created yet).

git-svn-id: https://svn.forgerock.org/opendj/trunk@10925 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     

22 Jul, 2014

2 commits

  • 46491d27   Code Review: Matthew Swift ... Browse Code » 
    
Pair.java:
Added COMPARATOR constant + getPairComparator() static method to return a Comparator of Pairs made of Comparable elements.

PairTestCase.java: ADDED


git-svn-id: https://svn.forgerock.org/opendj/trunk@10924 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • bd3ca8d8   Checkpoint commit for OPENDJ-1206 : Create a new ReplicationBackend/ChangelogBac… ... Browse Code » 
    …kend to support cn=changelog

Merged + corrected backend error messages by passing in the backendID into the error messages + fixed translations (!?!)
Started implementing ChangelogBackend by adding error messages.
Used Collections.emptySet() for supported controls and / or features, where applicable.
Generecized Backend class with the type of the configuration object + applied specific types to sub-classes of Backend.
Code cleanups + factorized code by extracting methods + ran AutoRefactor.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10923 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

21 Jul, 2014

7 commits

  • e4fdfa3d   Checkpoint commit for OPENDJ-1206 : Create a new ReplicationBackend/ChangelogBackend ... Browse Code » 
    to support cn=changelog
CR-4053

Preparatory work for the changelog backend:  add a new behavior to cursors, in order to be
 able to start cursor at a given key instead of starting just after. 
 In order to avoid introducing another boolean in the methods, I created two enums to 
 define the behavior of a cursor.

Note that JE implementation does not yet implement the new behavior.

* DBCursor class : add two enums KeyMatchingStrategy and PositionStrategy
** The new behavior corresponds to PositionStrategy.ON_MATCHING_KEY, which allow 
 to position on the record with the given key (while AFTER_MATCHING_KEY position 
 just after the record with the given key).

* ReplicationDomainDB : add PositionStrategy argument for all methods that returns a cursor

* ReplicationServerDomain : getCursorFrom(DN, ServerState) method calls underlying method
  with PositionStrategy.AFTER_MATCHING_KEY

* FileChangelogDB : add PositionStrategy argument for all methods that returns a cursor

* FileReplicaDB : add PositionStrategy argument to generateCursorFrom(CSN) method

* FileReplicaDBCursor : add PositionStrategy argument to the constructor, implement the
  new behavior when position strategy is ON_MATCHING_KEY

* DomainDBCursor, MultiDomainDBCursor : add PositionStrategy argument to the constructor, 
 pass the strategy to underlying cursors

* Log, LogFile, BlockLogReader : implement the new behavior when position strategy 
  is ON_MATCHING_KEY

* ChangeNumberIndexer : use AFTER_MATCHING_KEY strategy when retrieving the 
  cursor (no behavior change)

* JEChangelogDB : add PositionStrategy argument for all methods that returns a cursor, 
   but getCursorFrom(DN, int, CSN, PositionStrategy) method does NOT implement the
   new ON_MATCHING_KEY strategy.

* Update of tests classes to match method signature but no new tests added for the 
  new behavior (ON_MATCHING_KEY) - to be done later
  

git-svn-id: https://svn.forgerock.org/opendj/trunk@10922 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     
  • 1a5bcd85   Checkpoint commit for OPENDJ-1206 : Create a new ReplicationBackend/ChangelogBackend ... Browse Code » 
    to support cn=changelog
CR-4052

Revert previous changes corresponding to CR-4052 as a better approach is to implement 
the processing of  "Changelog Exchange Control" directly in the changelog backend.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10921 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     
  • 473bbe2f   Fix OPENDJ-1519: Incorrect check for deprecated password schemes ... Browse Code » 
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10919 41b1ffd8-f28e-4786-ab96-9950f0a78031
    cjr
     
  • 72f2668d   OPENDJ-1441 (CR-4037) Persistent searches on external changelog do not return ch… ... Browse Code » 
    …anges for new replicas and new domains


Fixed a problem introduced in r10912.
Problem was due to removing domains from the MultiDomainDBCursor:
In CompositeDBCursor.removeNoLongerNeededCursors(), code iterates over cursors and then forget baseDNs to remove only they match with a cursor.
Problem is that it should be the other way around: iterate over the baseDNs to remove and always forget them whether or not a matching cursor is found.



ChangeNumberIndexer.java:
In run(), better handled the removed domains.

CompositeDBCursor.java:
In removeNoLongerNeededCursors(), iterate over the baseDNs to remove, find a cursor and remove it if found, then always forget the baseDN. 
Added abstract method removedCursorsIterator().
Removed isCursorNoLongerNeededFor() and cursorRemoved().

DomainDBCursor.java, MultiDomainDBCursor.java, CompositeDBCursorTest.java:
Consequence of the changes to CompositeDBCursor.



ExternalChangeLogTest.java:
Code cleanup.
Extracted method readMessages() to factorize code.
Added method assertLastCookieDifferentThanLastValue() to loop until last cookie is updated.
Added inner class Results.

FileChangelogDB.java, JEChangelogDB.java:
In getExistingOrNewDomainMap(), only add the new domain if the baseDN is from an ECL enabled domain.

FileReplicaDBCursor.java:
Fixed javadoc.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10918 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 6dae0c0d   Checkpoint commit for OPENDJ-1206 : Create a new ReplicationBackend/ChangelogBackend ... Browse Code » 
    to support cn=changelog
CR-4052

Preparatory work for the changelog backend: make the "Changelog Exchange Control"
 available to the backend when search(SearchOperation) method is called 

* LocalBackendSearchOperation#handleRequestControls() method:
   handle the "Changelog Exchange Control" : add it to the SearchOperation 
   if provided in the request

* SearchOperation, SearchOperationBasis and SearchOperationWrapper classes:
   add getter and setter for the "Changelog Exchange Control" (ECLRequestControl class)

* core.properties : add new message for case when  "Changelog Exchange Control" is requested
 but not supported by a backend

git-svn-id: https://svn.forgerock.org/opendj/trunk@10917 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     
  • 9f352048   Checkpoint commit for OPENDJ-1206 : Create a new ReplicationBackend/ChangelogBackend ... Browse Code » 
    to support cn=changelog

 Skeleton of the new backend class : ChangelogBackend.java 

git-svn-id: https://svn.forgerock.org/opendj/trunk@10916 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     
  • 5a41afcc   Checkpoint commit for OPENDJ-1206 : Create a new ReplicationBackend/ChangelogBackend ... Browse Code » 
    to support cn=changelog
CR-4036

Preparatory work :  change implementation of virtual attribute providers related to external
 changelog to be independant of ECL workflow

*  ReplicationServer class:
** remove duplication of virtual attribute providers creation (for registering
    and deregistering)
** create virtual attribute providers by injecting the replication server instance

* FirstChangeNumberVirtualAttributeProvider, LastChangeNumberVirtualAttributeProvider, 
  LastCookieVirtualProvider classes:
** inject ReplicationServer in constructor in order to use it in getValues() method
** remove ConfigurationChangeListener implementation because no config change 
 can happen

* ChangelogBaseDNVirtualAttributeProvider class:
** remove ConfigurationChangeListener implementation because no config change 
 can happen

git-svn-id: https://svn.forgerock.org/opendj/trunk@10915 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     

19 Jul, 2014

1 commit

18 Jul, 2014

3 commits

  • 24850d2b   Fixed CompositeDBCursorTest.recycleTwoElementsCursor() test. ... Browse Code » 
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10913 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 10df020c   OPENDJ-1441 (CR-4037) Persistent searches on external changelog do not return ch… ... Browse Code » 
    …anges for new replicas and new domains



This commit only fixes the "new replicas" case.

For this, I made the existing CompositeDBCursor abstract and implemented 2 new cursors: DomainDBCursor and MultiDomainDBCursor which iterate on a single replication domain or in a cross domain fashion.

These new cursors are able to react to topology changes like a new domain or a new replica.
However, ECL persistent search cannot take advantage of them right now because their list of ECLServerHandler.domainCtxts is initialized at the start of the search and never updated after.


Please note that I added new methods to ReplicationDomainDB, but several of them are just called by DomainDBCursor, MultiDomainDBCursor or ChangeNumberIndexer which are considered internal classes to the changelog DB. So maybe should we find a way to hide them from code client to the changelogDB.

These changes meant I could remove the awful double way to use the CompositeDBCursor + I could also remove most cursor management from ChangeNumberIndexer. I think I might find a way in a subsequent commit to also get rid of ChangeNumberIndexer.replicasOffline.

Alas the same changes are duplicated in JE + file based changelog. 



DomainDBCursor.java, MultiDomainDBCursor.java: ADDED

ReplicationDomainDB.java:
Added getCursorFrom(MultiDomainServerState startAfterState) and unregisterCursor(DBCursor) called by DomainDBCursor and MultiDomainDBCursor.

FileChangelogDB.java, JEChangelogDB.java:
Added registeredDomainCursors and registeredMultiDomainCursors fields.
In getExistingOrNewDomainMap(), updated MultiDomainDBCursors when a new domain is created.
In getCursorFrom(DN baseDN, int serverId, CSN), created the ReplicaOfflineCSN there from getCursorFrom(DN baseDN, ServerState).
Added newDomainDBCursor().
Reworked newOfflineCSN().
Implemented new methods in ReplicationDomainDB.
In getOrCreateReplicaDB(), updated DomainDBCursors when a new replica is created.
Synchronized the two files to ease diffing them together.

ChangeNumberIndexer.java:
Removed the responsibility to manage cursors from this class.
Removed allCursors, newCursors fields.
In publishUpdateMsg(), initialize(), moveForwardMediumConsistencyPoint() and run() removed code that dealt with creating/opening/recreating/removing cursors.
Removed resetNextChangeForInsertDBCursor(), ensureCursorExists(), removeCursors(), getCursor(), recycleExhaustedCursors(), createNewCursors(), getPrecedingCSN().
Made getPrecedingCSN() public static.
Added logUnexpectedException().


CompositeDBCursor.java:
Now abstract.
Removed ctor.
Removed recycleExhaustedCursors field.
Added incorporateNewCursors, isCursorNoLongerNeededFor(), cursorRemoved() and addCursor() + used them in next().
In next(), extracted recycleExhaustedCursors + used newly added removeNoLongerNeededCursors().
In close(), completed code.

ChangeNumberIndexerTest.java:
Consequence of the changes to ChangeNumberIndexer.
Renamed cursors field to replicaDBCursors.
Added multiDomainCursor, domainDBCursors fields.
Added eclEnabledDomains field to separate it from startCNIndexer().
Changed setup() and addReplica().

CompositeDBCursorTest.java:
Consequence of the change to CompositeDBCursor.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10912 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • ec19c95f   Fix OPENDJ-1516: Trunk sends MSG_TYPE_REPLICA_OFFLINE messages to a v7 DS ... Browse Code » 
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10910 41b1ffd8-f28e-4786-ab96-9950f0a78031
    cjr
     

16 Jul, 2014

1 commit

15 Jul, 2014

1 commit

10 Jul, 2014

3 commits

09 Jul, 2014

1 commit

08 Jul, 2014

1 commit

07 Jul, 2014

4 commits

  • 4b38c4f4   Fix unit-test breakage due to changes with the isEqualsTo method. ... Browse Code » 
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10870 41b1ffd8-f28e-4786-ab96-9950f0a78031
    ludo
     
  • 99bef4e4   Fixed NPE on querying cn=changelog with persistent search. ... Browse Code » 
    Regression was introduced in r10655.

ECLServerHandler.java:
In buildDomainContexts(), fixed the code to match the code behaviour pre r10655.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10869 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • fa189ecd   OPENDJ-1453 Replica offline messages should be synced with updates ... Browse Code » 
    In r10840, the change to JE/FileChangelogDB.getCursorFrom(DN, ServerState) unnecessarily triggers a lot of calls to the underlying DB (JE or file based) to retrieve the ChangelogState.
As an optimization, keeping an in-memory version of the ChangelogState in synch with the on-disk version will help.


ChangelogState.java:
Now thread safe.
Added removeOfflineReplica(), isEqualTo().
Changed domainToServerIds from Map<DN, List<Integer>> to Map<DN, Set<Integer>>.
In getOfflineReplicas(), now return a MultiDomainServerState.

ChangeNumberIndexer.java:
Consequence of the changes to ChangelogState.


ReplicationDbEnv.java:
Added changelogState field, updated at the same time as the on-disk changelogstate DB
Added getChangelogState(), called by client code instead of readChangelogState().
Renamed readChangelogState() to private readOnDiskChangelogState().
Added stateLock field to sync updates to in-memory and on-disk changelog state.

ReplicationEnvironment.java:
Added changelogState field, updated at the same time as the on-disk changelogstate DB
Added getChangelogState(), called by client code instead of readChangelogState().
Renamed readChangelogState() to private readOnDiskChangelogState().
Renamed domainLock field to domainsLock.

replication.properties:
Removed now unused error message.

FileChangelogDB.java, JEChangelogDB.java:
Consequence of the changes to ChangelogState and ReplicationEnvironment/ReplicationDbEnv.


MultiDomainServerState.java, ServerState.java:
Added getSnapshot() for unit tests.

ReplicationEnvironmentTest.java, ReplicationDbEnvTest.java:
Consequence of the changes to ReplicationEnvironment and ChangelogState.
Used the fake server.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10868 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 7a62dbf2   OPENDJ-1509 (CR-3963) The opendj-ldap-sdk is redundant and can be removed ... Browse Code » 
    Remove the opendj-ldap-sdk project and update dependencies in all pom files and in the project documentation.
* all pom.xml
** Removing reference to the opendj-ldap-sdk project and add dependencies to opendj-core and openj-grizzly
* opendj-core/.../index.xml.vm
** Updating the opendj-core documentation with the opendj-ldap-sdk old documentation page
* opendj-project site.xml chap-get-sdk.xml
** Updating project documentation menu and developer guide

git-svn-id: https://svn.forgerock.org/opendj/trunk@10867 41b1ffd8-f28e-4786-ab96-9950f0a78031
    gaetan
     

04 Jul, 2014

3 commits

03 Jul, 2014

4 commits

  • 6f6d1e62   OPENDJ-1453 (CR-3938) Replica offline messages should be synced with updates ... Browse Code » 
    This changed managed to solve the combined DS/RS case. Mostly, because robot test is sometimes failing yet.
This is fixed by introducing a DSRSShutdownSync class which ensures that when the DS sends a ReplicaOfflineMsg, the RS will relay it to the rest of the topology.
Second main change is to ensure ReplicaOfflineMsgs do not update the domain's ServerState otherwise the functional tests detect an inconsistency in the backend ds-sync-state data between the 2 replica: the one that is stopped and the one that is never stopped. For this I added UpdateMsg.contributesToDomainState() and used it throughout.
The rest of the change is passing the DSRSShutdownSync down method calls.



DSRSShutdownSync.java: ADDED

UpdateMsg.java, ReplicaOfflineMsg.java:
Added contributesToDomainState()



MultimasterReplication.java:
Added dsrsShutdownSync field + created it here + passed it down creation of LDAPReplicationDomain and ReplicationServerListener.
Code cleanup.

LDAPReplicationDomain.java:
Added dsrsShutdownSync field + used it in publishReplicaOfflineMsg().

ReplicationServerListener.java
Added dsrsShutdownSync field + used it when creating ReplicationServer.
Code cleanup.

ReplicationServer.java:
Added dsrsShutdownSync field + added getDSRSShutdownSync() getter.

ServerHandler.java
Called ReplicationServer.getDSRSShutdownSync() when creating ServerWriter.
Code cleanup.

ServerWriter.java:
Added dsrsShutdownSync field + used it in run().



ECLServerWriter.java:
Consequence of the change to ServerWriter.
Code cleanup.



RemotePendingChanges.java, ReplicationDomain.java, MessageHandler.java:
Used UpdateMsg.contributesToDomainState().

git-svn-id: https://svn.forgerock.org/opendj/trunk@10861 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 115c364e   Code cleanup. ... Browse Code » 
    Made use of constants and serverConstants where applicable.
Fixed typos.
Simplified tests.
Fixed a couple of erroneous impossible condition tests.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10860 41b1ffd8-f28e-4786-ab96-9950f0a78031
    ludo
     
  • 89512276   Factorized code into CommonArguments.getSearchScope(). ... Browse Code » 
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10859 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 60213812   Code cleanup. ... Browse Code » 
    
ReplicationServerDomain.java:
In put(), renamed update parameter to updateMsg.
Extracted methods getPreparedAssuredInfo(), getAssuredServers(), isDifferentGenerationId(), isUpdateMsgFiltered().
Dramatically simplified addUpdate().

ServerReader.java, ServerWriter.java:
Extracted method isUpdateMsgFiltered().

git-svn-id: https://svn.forgerock.org/opendj/trunk@10858 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac