20 Aug, 2014

1 commit


19 Aug, 2014

9 commits

  • Implemented PositionStrategy.ON_MATCHING_KEY for JEChangelogDB.getCursorFrom(DN baseDN, int serverId).
    
    
    
    JEChangelogDB.java, JEReplicaDB.java:
    Implemented PositionStrategy.ON_MATCHING_KEY.
    
    JEReplicaDBCursor.java:
    Added new field positionStrategy.
    Simplified code in next() + added gotoNext().
    
    ReplicationDB.java:
    In openReadCursor(), added PositionStrategy parameter.
    Made createReplicationKey() handle null CSNs.
    Inner class ReplServerDBCursor now implement DBCursor + Added getRecord() + Made next(), now return boolean instead of UpdateMsg + Extracted method computeCurrentRecord().
    Added new field "UpdateMsg currentRecord".
    Added PositionStrategy parameter to ReplServerDBCursor ctor + protected special code for PositionStrategy.AFTER_MATCHING_KEY.
    
    
    JEReplicaDBTest.java:
    Added test cases for new PositionStrategy ON_MATCHING_KEY.
    Split assertFoundInOrder() in two: one each for AFTER_MATCHING_KEY and ON_MATCHING_KEY.
    In testGenerateCursorFrom(), changed the code to cover all CSNs.
    Extracted method assertNextCSN() and shutdown().
    Used SoftAssertions.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10964 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • git-svn-id: https://svn.forgerock.org/opendj/trunk@10961 41b1ffd8-f28e-4786-ab96-9950f0a78031
    mark
     
  • git-svn-id: https://svn.forgerock.org/opendj/trunk@10959 41b1ffd8-f28e-4786-ab96-9950f0a78031
    cjr
     
  • …anges for new replicas and new domains
    
    Follow up of r10944 (and r10912 originally).
    Update ReplicaCursors with new replica offline information (For example, when replica came online or when there is more up to date replica offline information).
    Renamed ReplicaOfflineCursor(Test) to ReplicaCursor(Test)
    
    
    
    FileChangelogDB.java, JEChangelogDB.java:
    Added replicaCursors Map field.
    In getCursorFrom(), registered replicaCursors.
    In unregisterCursor(), handled ReplicaCursors.
    In notifyReplicaOnline() and notifyReplicaOffline(), called new method updateCursorsWithOfflineCSN().
    
    ReplicaOfflineCursor.java: RENAMED to ReplicaCursor
    Removed returnReplicaOfflineMsg field.
    Changed replicaOfflineMsg field from ReplicaOfflineMsg to AtomicReference<ReplicaOfflineMsg>.
    In next(), rewrote the code to support resetting the replica offline CSN + added isReplicaOfflineMsgOutdated() and setOfflineCSN()
    Added replicaID and domainDB fields + called ReplicationDomainDB.unregisterCursor() in close() + added getReplicaID().
    
    ReplicaOfflineCursorTest.java: RENAMED to ReplicaOfflineCursorTest
    Consequence of the change to ReplicaCursor.
    
    ChangeNumberIndexer.java:
    Removed outdated TODO
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10958 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • …anges for new replicas and new domains
    
    Fixing a deadlock situation with ExternalChangeLogTest.
    When removing domains, the cursors were not properly closed by ChangeNumberIndexer.run() before calling Object.wait().
    Relied on the fact that DBCursors are not thread safe so if client code calls MultiDomainDBCursor.removeDomain(), then the cursor is immediately removed.
    I did not see the problem before because unit tests on trunk execute by default with file based changelog db.
    
    
    ChangeNumberIndexer.java:
    In run(), simplified the code that removes the domain cursors.
    
    CompositeDBCursor.java:
    Removed removeNoLongerNeededCursors() and removedCursorsIterator() + extracted method removeCursor() from it.
    Added another removeCursor() which also removes from the exhaustedCursors.
    
    MultiDomainDBCursor.java
    Consequence of the change to CompositeDBCursor.
    Removed removeDomains field and removedCursorsIterator().
    In removeDomain(), now call CompositeDBCursor.removeCursor().
    
    DomainDBCursor.java:
    Consequence of the change to CompositeDBCursor.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10955 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • … create a new suffix on existing backend
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10954 41b1ffd8-f28e-4786-ab96-9950f0a78031
    mark
     
  • git-svn-id: https://svn.forgerock.org/opendj/trunk@10952 41b1ffd8-f28e-4786-ab96-9950f0a78031
    mark
     
  • git-svn-id: https://svn.forgerock.org/opendj/trunk@10950 41b1ffd8-f28e-4786-ab96-9950f0a78031
    mark
     
  • git-svn-id: https://svn.forgerock.org/opendj/trunk@10947 41b1ffd8-f28e-4786-ab96-9950f0a78031
    mark
     

12 Aug, 2014

1 commit

  • In ctor, removed useless code, because this case is already covered in next().
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10942 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

11 Aug, 2014

1 commit

  • PBKDF2PasswordStorageScheme.java, PKCS5S2PasswordStorageScheme.java:
    Fixed a bug in PBKF2 + PKCS5S2 encodeOffline() methods which did not know how to deal with a byte[]. The right way was to wrap it in a ByteString like SaltedSHA1PasswordStorageScheme was already doing. BTW these encodeOffline() methods are never called from production code.
    Did more code factorization.
    Removed dead code in both with plainPlusSalt local variable which was not used for anything.
    Fixed a few bugs in PKCS5S2 where locks where not properly taken (or carefully avoided?).
    Made both PBKF2 + PKCS5S2 code look very similar now.
    
    PBKDF2PasswordStorageSchemeTestCase.java, PKCS5S2PasswordStorageSchemeTestCase.java, PasswordStorageSchemeTestCase.java, SaltedSHA1PasswordStorageSchemeTestCase.java, SaltedSHA256PasswordStorageSchemeTestCase.java, SaltedSHA384PasswordStorageSchemeTestCase.java, SaltedSHA512PasswordStorageSchemeTestCase.java:
    Removed dead code.
    
    CryptPasswordStorageSchemeTestCase.java:
    Made it reuse more code from PasswordStorageSchemeTestCase.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10941 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

08 Aug, 2014

1 commit


07 Aug, 2014

1 commit


30 Jul, 2014

2 commits


29 Jul, 2014

1 commit


25 Jul, 2014

4 commits

  • Adding two new parameters to the xxxrate tools, maximum duration time and warm-up duration time.
    
    ** PerformanceRunner.java
    * Adding a timer thread for the max duration feature and add warm-up implementation.
    ** AddRate.java SearchRate.java
    * Adding resetStats method to reset local statistics (i.e extra columns)
    ** tools.properties
    * Adding warm-up message
    ** AuthRateITCase.java
    ** Adding a test with warm-up phase
     
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10932 41b1ffd8-f28e-4786-ab96-9950f0a78031
    gaetan
     
  • PBKDF2PasswordStorageSchemeTestCase.java: ADDED
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10931 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • …anges for new replicas and new domains
    
    Created ECLMultiDomainDBCursor to only let ECL enabled domains reach the ChangeNumberIndexer.
    
    ChangeNumberIndexer.java:
    Wrapped the MultiDomainDBCursor into an ECLMultiDomainDBCursor.
    
    ECLMultiDomainDBCursor.java: ADDED
    
    JEChangelogDB.java, FileChangelogDB.java:
    Removed unnecessary calls to MultimasterREplication.isECLEnabledDomain().
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10930 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Used AutoRefactor.
    Reduced variable scopes.
    
    PBKDF2PasswordStorageScheme.java:
    Extracted methods getDigestBytes(), encodeAndMatch(), 
    Removed unnecessary use of StringBuilders.
    In passwordMatches() and authPasswordMatches(), used String.indexOf().
    
    PKCS5S2PasswordStorageScheme.java:
    See general changes.
    
    PKCS5S2PasswordStorageSchemeTestCase.java:
    In getTestPasswords(), reused the super method and filtered entries.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10929 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

23 Jul, 2014

4 commits

  • …anges for new replicas and new domains
    
    Code Review: Nicolas Capponi
    
    Fixed regression introduced in r10912. It made fractional replication tests fail.
    ChangeNumberIndexer.initialize() must ignore ReplicaOfflineMsgs, just like ChangeNumberIndexer.run().
    
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10928 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Adding an add/del rate tool named addrate.
    -opendj-core
        ** EntryGenerator.java:
        * Changing the subtemplate parsing behavior, add the "generated branches" options
        ** TemplateFile.java
        * Adding the "generated branches" options
        ** addrate.template
        * New template used to generate entries
    
    -opendj-ldap-toolkit
        ** addrate (bat and bin)
        * Adding new tool scripts for unix and windows
        ** AddRate.java
        * addrate tool new class
        ** PerformanceRunner.java
        * Using parameter object in constructor, adding some messages localization
        ** PerformanceRunnerOptions.java
        * Helps to simplify the way to initialize a PerformanceRunner instance
        ** MakeLDIF.java
        * Little refactoring in order to allow the addrate tool to use some make-ldif parameters
        ** tools.properties
        * Adding addrate tool message and also some new generic messages
        ** Minor changes (constructor) in other tools and tests files
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10927 41b1ffd8-f28e-4786-ab96-9950f0a78031
    gaetan
     
  • …anges for new replicas and new domains
    
    Fix done in r10925 ported to JE changelog implementation
      
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10926 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     
  • …anges for new replicas and new domains
    
    Fixed a problem introduced in r10918.
    
    In getExistingOrNewDomainMap(DN) method, the MultimasterReplication.isECLEnabledDomain() method is called.
    However, this method blocks forever at replication startup because MultimasterReplication state is equals to STARTING
    and is only set to RUNNING in code that occurs after in the same thread.
    
    Code sequence is :
    1. SynchronizationProviderConfigManager calls MultimasterReplication.initializeSynchronizationProvider()
       try to call MultimasterReplication.isECLEnabledDomain() but MultimasterReplication state is equals 
       to STARTING so it waits indefinitely on a state change
    2. SynchronizationProviderConfigManager calls DirectoryServer.registerSynchronizationProvider(provider)
       MultimasterReplication state is set to RUNNING
    
    Add a check on registeredMultiDomainCursors to ensure that isECLEnabledDomain() is not called when 
    the replication server is starting (as no cursors can be created yet).
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10925 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     

22 Jul, 2014

2 commits

  • 
    Pair.java:
    Added COMPARATOR constant + getPairComparator() static method to return a Comparator of Pairs made of Comparable elements.
    
    PairTestCase.java: ADDED
    
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10924 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • …kend to support cn=changelog
    
    Merged + corrected backend error messages by passing in the backendID into the error messages + fixed translations (!?!)
    Started implementing ChangelogBackend by adding error messages.
    Used Collections.emptySet() for supported controls and / or features, where applicable.
    Generecized Backend class with the type of the configuration object + applied specific types to sub-classes of Backend.
    Code cleanups + factorized code by extracting methods + ran AutoRefactor.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10923 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

21 Jul, 2014

7 commits

  • to support cn=changelog
    CR-4053
    
    Preparatory work for the changelog backend:  add a new behavior to cursors, in order to be
     able to start cursor at a given key instead of starting just after. 
     In order to avoid introducing another boolean in the methods, I created two enums to 
     define the behavior of a cursor.
    
    Note that JE implementation does not yet implement the new behavior.
    
    * DBCursor class : add two enums KeyMatchingStrategy and PositionStrategy
    ** The new behavior corresponds to PositionStrategy.ON_MATCHING_KEY, which allow 
     to position on the record with the given key (while AFTER_MATCHING_KEY position 
     just after the record with the given key).
    
    * ReplicationDomainDB : add PositionStrategy argument for all methods that returns a cursor
    
    * ReplicationServerDomain : getCursorFrom(DN, ServerState) method calls underlying method
      with PositionStrategy.AFTER_MATCHING_KEY
    
    * FileChangelogDB : add PositionStrategy argument for all methods that returns a cursor
    
    * FileReplicaDB : add PositionStrategy argument to generateCursorFrom(CSN) method
    
    * FileReplicaDBCursor : add PositionStrategy argument to the constructor, implement the
      new behavior when position strategy is ON_MATCHING_KEY
    
    * DomainDBCursor, MultiDomainDBCursor : add PositionStrategy argument to the constructor, 
     pass the strategy to underlying cursors
    
    * Log, LogFile, BlockLogReader : implement the new behavior when position strategy 
      is ON_MATCHING_KEY
    
    * ChangeNumberIndexer : use AFTER_MATCHING_KEY strategy when retrieving the 
      cursor (no behavior change)
    
    * JEChangelogDB : add PositionStrategy argument for all methods that returns a cursor, 
       but getCursorFrom(DN, int, CSN, PositionStrategy) method does NOT implement the
       new ON_MATCHING_KEY strategy.
    
    * Update of tests classes to match method signature but no new tests added for the 
      new behavior (ON_MATCHING_KEY) - to be done later
      
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10922 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     
  • to support cn=changelog
    CR-4052
    
    Revert previous changes corresponding to CR-4052 as a better approach is to implement 
    the processing of  "Changelog Exchange Control" directly in the changelog backend.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10921 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     
  • git-svn-id: https://svn.forgerock.org/opendj/trunk@10919 41b1ffd8-f28e-4786-ab96-9950f0a78031
    cjr
     
  • …anges for new replicas and new domains
    
    
    Fixed a problem introduced in r10912.
    Problem was due to removing domains from the MultiDomainDBCursor:
    In CompositeDBCursor.removeNoLongerNeededCursors(), code iterates over cursors and then forget baseDNs to remove only they match with a cursor.
    Problem is that it should be the other way around: iterate over the baseDNs to remove and always forget them whether or not a matching cursor is found.
    
    
    
    ChangeNumberIndexer.java:
    In run(), better handled the removed domains.
    
    CompositeDBCursor.java:
    In removeNoLongerNeededCursors(), iterate over the baseDNs to remove, find a cursor and remove it if found, then always forget the baseDN. 
    Added abstract method removedCursorsIterator().
    Removed isCursorNoLongerNeededFor() and cursorRemoved().
    
    DomainDBCursor.java, MultiDomainDBCursor.java, CompositeDBCursorTest.java:
    Consequence of the changes to CompositeDBCursor.
    
    
    
    ExternalChangeLogTest.java:
    Code cleanup.
    Extracted method readMessages() to factorize code.
    Added method assertLastCookieDifferentThanLastValue() to loop until last cookie is updated.
    Added inner class Results.
    
    FileChangelogDB.java, JEChangelogDB.java:
    In getExistingOrNewDomainMap(), only add the new domain if the baseDN is from an ECL enabled domain.
    
    FileReplicaDBCursor.java:
    Fixed javadoc.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10918 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • to support cn=changelog
    CR-4052
    
    Preparatory work for the changelog backend: make the "Changelog Exchange Control"
     available to the backend when search(SearchOperation) method is called 
    
    * LocalBackendSearchOperation#handleRequestControls() method:
       handle the "Changelog Exchange Control" : add it to the SearchOperation 
       if provided in the request
    
    * SearchOperation, SearchOperationBasis and SearchOperationWrapper classes:
       add getter and setter for the "Changelog Exchange Control" (ECLRequestControl class)
    
    * core.properties : add new message for case when  "Changelog Exchange Control" is requested
     but not supported by a backend
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10917 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     
  • to support cn=changelog
    
     Skeleton of the new backend class : ChangelogBackend.java 
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10916 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     
  • to support cn=changelog
    CR-4036
    
    Preparatory work :  change implementation of virtual attribute providers related to external
     changelog to be independant of ECL workflow
    
    *  ReplicationServer class:
    ** remove duplication of virtual attribute providers creation (for registering
        and deregistering)
    ** create virtual attribute providers by injecting the replication server instance
    
    * FirstChangeNumberVirtualAttributeProvider, LastChangeNumberVirtualAttributeProvider, 
      LastCookieVirtualProvider classes:
    ** inject ReplicationServer in constructor in order to use it in getValues() method
    ** remove ConfigurationChangeListener implementation because no config change 
     can happen
    
    * ChangelogBaseDNVirtualAttributeProvider class:
    ** remove ConfigurationChangeListener implementation because no config change 
     can happen
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10915 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     

19 Jul, 2014

1 commit


18 Jul, 2014

3 commits

  • git-svn-id: https://svn.forgerock.org/opendj/trunk@10913 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • …anges for new replicas and new domains
    
    
    
    This commit only fixes the "new replicas" case.
    
    For this, I made the existing CompositeDBCursor abstract and implemented 2 new cursors: DomainDBCursor and MultiDomainDBCursor which iterate on a single replication domain or in a cross domain fashion.
    
    These new cursors are able to react to topology changes like a new domain or a new replica.
    However, ECL persistent search cannot take advantage of them right now because their list of ECLServerHandler.domainCtxts is initialized at the start of the search and never updated after.
    
    
    Please note that I added new methods to ReplicationDomainDB, but several of them are just called by DomainDBCursor, MultiDomainDBCursor or ChangeNumberIndexer which are considered internal classes to the changelog DB. So maybe should we find a way to hide them from code client to the changelogDB.
    
    These changes meant I could remove the awful double way to use the CompositeDBCursor + I could also remove most cursor management from ChangeNumberIndexer. I think I might find a way in a subsequent commit to also get rid of ChangeNumberIndexer.replicasOffline.
    
    Alas the same changes are duplicated in JE + file based changelog. 
    
    
    
    DomainDBCursor.java, MultiDomainDBCursor.java: ADDED
    
    ReplicationDomainDB.java:
    Added getCursorFrom(MultiDomainServerState startAfterState) and unregisterCursor(DBCursor) called by DomainDBCursor and MultiDomainDBCursor.
    
    FileChangelogDB.java, JEChangelogDB.java:
    Added registeredDomainCursors and registeredMultiDomainCursors fields.
    In getExistingOrNewDomainMap(), updated MultiDomainDBCursors when a new domain is created.
    In getCursorFrom(DN baseDN, int serverId, CSN), created the ReplicaOfflineCSN there from getCursorFrom(DN baseDN, ServerState).
    Added newDomainDBCursor().
    Reworked newOfflineCSN().
    Implemented new methods in ReplicationDomainDB.
    In getOrCreateReplicaDB(), updated DomainDBCursors when a new replica is created.
    Synchronized the two files to ease diffing them together.
    
    ChangeNumberIndexer.java:
    Removed the responsibility to manage cursors from this class.
    Removed allCursors, newCursors fields.
    In publishUpdateMsg(), initialize(), moveForwardMediumConsistencyPoint() and run() removed code that dealt with creating/opening/recreating/removing cursors.
    Removed resetNextChangeForInsertDBCursor(), ensureCursorExists(), removeCursors(), getCursor(), recycleExhaustedCursors(), createNewCursors(), getPrecedingCSN().
    Made getPrecedingCSN() public static.
    Added logUnexpectedException().
    
    
    CompositeDBCursor.java:
    Now abstract.
    Removed ctor.
    Removed recycleExhaustedCursors field.
    Added incorporateNewCursors, isCursorNoLongerNeededFor(), cursorRemoved() and addCursor() + used them in next().
    In next(), extracted recycleExhaustedCursors + used newly added removeNoLongerNeededCursors().
    In close(), completed code.
    
    ChangeNumberIndexerTest.java:
    Consequence of the changes to ChangeNumberIndexer.
    Renamed cursors field to replicaDBCursors.
    Added multiDomainCursor, domainDBCursors fields.
    Added eclEnabledDomains field to separate it from startCNIndexer().
    Changed setup() and addReplica().
    
    CompositeDBCursorTest.java:
    Consequence of the change to CompositeDBCursor.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10912 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • git-svn-id: https://svn.forgerock.org/opendj/trunk@10910 41b1ffd8-f28e-4786-ab96-9950f0a78031
    cjr
     

16 Jul, 2014

1 commit


15 Jul, 2014

1 commit