05 Nov, 2013

3 commits

  • In createChangelogEntry(), removed duplicate parameters.
    In createEntryFromMsg(), extracted method addAttribute().
    Changed returnECLControl field to boolean (incorrectly was Boolean)
    Improved formatting.
    Removed comments paraphrasing the code.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9777 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Added complimentary javadocs to explain what these constants mean.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9776 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Renamed variables and changed comments to match the new terminology.
    Added more comments to explain what the code is doing.
    
    ReplicationBroker.java
    Renamed replicationServerInfo => rsInfo.
    
    replication.properties
    Removed space characters at the end of each lines.
    
    RSInfo.java:
    In toString(), removed newline + improved formatting.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9775 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

04 Nov, 2013

4 commits

  • …d (RSUpdater) can spin
    
    
    This change is linked to the misbehaving RSUpdater thread which (wrongly):
    - could be started multiple times
    - would not shutdown willingly when the server is shutting down
    - would try to look for replay operations in the future
    
    
    
    LDAPReplicationDomain.java:
    In now(), ensured now() will always come last with isNewerThan() test.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9769 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • …d (RSUpdater) can spin
    
    
    This change is linked to the misbehaving RSUpdater thread which (wrongly):
    - could be started multiple times
    - would not shutdown willingly when the server is shutting down
    - would try to look for replay operations in the future
    
    
    
    LDAPReplicationDomain.java:
    In buildAndPublishMissingChanges(), only exit the loop when the currentStartCSN is newer than now (instead of the incorrect currentStartCSN + 10s).
    
    HistoricalCsnOrderingTest.java:
    Code cleanup.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9767 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • git-svn-id: https://svn.forgerock.org/opendj/trunk@9765 41b1ffd8-f28e-4786-ab96-9950f0a78031
    cjr
     
  • …d (RSUpdater) can spin
    
    
    This change is linked to the misbehaving RSUpdater thread which (wrongly):
    - could be started multiple times
    - would not shutdown willingly when the server is shutting down
    - would try to look for replay operations in the future
    
    
    
    LDAPReplicationDomain.java:
    Added AtomicReference for RSUpdater thread.
    Used DirectoryThread.isShutdownInitiated() for ServerStateFlush and RSUpdater threads.
    In RSUpdater, added shutdown field to pass it down to buildAndPublishMissingChanges() + overrode initiateShutdown() to set the shutdown field to true.
    In shutdown(), called initiateShutdown() for RSUpdater and RSUpdater threads.
    In sessionInitiated(), only start the RSUpdaterThread if it does not already exist.
    In buildAndPublishMissingChanges(), added early exits in case of shutdown.
    Added now() method.
    
    HistoricalCsnOrderingTest.java:
    Consequence of the change to LDAPReplicationDomain.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9764 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

30 Oct, 2013

2 commits

  • In runTask(), improved error logging in case of exceptions.
    Little code cleanup.
    
    TaskThread.java
    Logged the task state after the task finished.
    Comments => javadocs.
    
    backend*.properties:
    Updated NOTICE_TASK_FINISHED_414 message. Updated the translations where I could, otherwise removed them (alas).
    
    replication.properties, LDAPReplicationDomain.java:
    Removed useless spaces in error logging
    
    DSInfo.java:
    In toString(), removed newline + improved formatting.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9762 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Added cloneWithReplicationServerId().
    
    LightweightServerHandler.java:
    Simplified ctor by just accepting a DSInfo object.
    Extracted method debugInfo()
    Removed redundant fields.
    
    ReplicationServerHandler.java:
    In processTopoInfoFromRS(), called DSInfo.cloneWithReplicationServerId() + consequence of the change to LightweightServerHandler.
    
    ReplicationDomain.java:
    Small code cleanup.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9761 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

28 Oct, 2013

2 commits

  • replication.properties
    Removed the forbidden final dot from i18n message.
    
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9755 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • …r in order to diagnose the causes of these events
    
    
    This change records how each replication server is evaluated by computeBestReplicationServer().
    It also logs why a DS switches from one RS to another RS.
    
    
    ReplicationBroker.java:
    Added static classes RSEvaluations and LocalEvaluation.
    Moved keepBest() method to RSEvaluations.
    In filterServersWithAllLocalDSChanges(), reworked the code to be more understandable + Added method getCSN() and rejectAllWithRSIsLaterThanBestRS(), rejectAllWithRSOnDifferentVMThanDS().
    In computeBestServerWhenConnected(), extracted method isServerOverloadingRS().
    
    replication.properties:
    Added tons of messages.
    
    ComputeBestServerTest.java:
    Moved this class to package org.opends.server.replication.service to match the package of the tested code.
    Consequence of the changes to ReplicationBroker.
    Added methods containsOnly(), getDescription(), getEval1() and getEval2().
    Reformatted the code a bit.
    Removed @throws from javadocs.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9753 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

25 Oct, 2013

1 commit


23 Oct, 2013

2 commits

  • Thanks to Chris Ridd for reviewing the patch I posted to the issue and letting me know by IM.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9747 41b1ffd8-f28e-4786-ab96-9950f0a78031
    mark
     
  • Replaced instance fields with directly storing and using the ReplicationDomainCfg object.
    
    ReplicationBroker.java:
    Replaced instance fields with directly storing and using the ReplicationDomainCfg object.
    Removed start(Set<String>), setChangeTimeHeartbeatInterval() (now useless), and setGroupId() (never used).
    Added getBaseDN(), getReplicationServerUrls(), getGroupId().
    In several methods, used Map.entrySet() instead of Map.keySet() with Map.get(Object)  + renamed replicationServerInfo to rsInfo.
    
    replication.properties:
    Removed now unused message.
    
    
    ReplicationDomain.java:
    Replaced instance fields with directly storing and using the ReplicationDomainCfg object.
    Code cleanups:
    - Removed useless use of "this." for method calls.
    - In processErrorMsg() and processUpdateDone(), collapsed if statements
    - Increased vertical density
    - Extracted method getGenId()
    - In waitForAckIfAssuredEnabled(), used early exit.
    
    
    DomainFakeCfg.java:
    Comments => javadocs
    baseDn => baseDN
    Removed several unused methods.
    Added setWindowSize() + instance member.
    
    *Test*.java
    Used the DomainFakeCfg everywhere + SortedSet instead of Set.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9746 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

21 Oct, 2013

1 commit


17 Oct, 2013

3 commits

  • 
    ReplicationDbEnv.java, replication.properties:
    In openDatabase(), fixed TODOs for i18n.
    In newErrorMessage(), improved error logging by including the DB name.
    
    JEChangelogDB.java:
    Simplified the code by calling domainToReplicaDBs.remove() from inside shutdownReplicaDBs().
    In shutdownReplicaDBs(), do not bother cleaning up the domainMap since it will be removed from domainToReplicaDBs.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9737 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • …e directory server's broker was not connected to a replication domain, hence it was impossible to add the changes to the changelogDB and then find them when querying the changelogDB.
    
    
    GenerationIdTest.java:
    Renamed assertConnectedToReplicationDomain() to waitConnectionToReplicationDomain().
    
    LDAPReplicationDomain.java:
    Removed uses of StringBuilder, relied on *Operation.toString() to do it for us.
    
    ReplicationBroker.java:
    Improved toString().
    
    ReplicationTestCase.java:
    Removed a wrong comment.
    Used Assertions.assertThat().
    
    ReplicationServerTest.java:
    Removed a useless call to DirectoryServer.getSynchronizationProviders().
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9736 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • In toString(), improved the returned String.
    Changed rsServerId field from Integer to int.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9731 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

16 Oct, 2013

1 commit

  • Used StaticUtils.stackTraceToSingleLineString(e)) for logging on one line only.
    
    ReplicationBackend.java:
    Renamed all *containers to *domains.
    Reworked findExportContainers() and findSearchContainers() into only one method: selectReplicationDomains().
    
    ListenerThread.java:
    Fixed formatting.
    
    ReplicationServerLoadBalancingTest.java:
    Code cleanup.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9729 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

15 Oct, 2013

1 commit

  • UpdateOperationTest.java:
    Added consumeAllMessages(ReplicationBroker).
    In csnGeneratorAdjust(), called consumeAllMessages().
    Extracted local variables serverId.
    Fixed formatting.
    Increased vertical density.
    Removed a useless call to Thread.sleep().
    Removed useless comments.
    
    ReplicationBroker.java:
    Used local variables "localSession" to avoid concurrency issues when setSession(null) is called by another thread.
    
    *.java:
    Implemented toString().
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9725 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

12 Oct, 2013

1 commit


11 Oct, 2013

4 commits

  • In computeInitialServerStatus(), used early exits.
    In computeBestServerForWeight(), extracted methods computeBestServerWhenNotConnected() and computeBestServerWhenConnected().
    Changed replicationServerUrls from Collection<String> to Set<String>.
    Removed useless field initialization to null.
    Renamed _publish() to publish() + reduced local variables scope.
    In receive(), renamed local variable replicationServerID to previousRsServerID and used this one more rather than the field.
    In changeConfig(), used Set.equals().
    Changed getReplicationMonitor() to getReplicationMonitorInstanceName().
    
    ReplicationDomain.java:
    Consequence of the change to ReplicationBroker.getReplicationMonitor(), inlined getReplicationMonitorInstanceName().
    Consequence of the change to ReplicationBroker.replicationServerUrls.
    
    TestCaseUtils.java:
    Moved newSet(), newSortedSet(), newList() here from test classes.
    
    *Test.java:
    Consequence of the changes to ReplicationBroker.replicationServerUrls.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9712 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • package org.opends.server.types requires javadoc even for private methods.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9711 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • RDN.java:
    In getDNValue(), changed parameter from String to ByteString.
    In compareTo(), reordered the if statements + extracted method compare().
    Comments => javadocs
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9710 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Fixed a regression introduced in r9605:  It was found by Christophe Sovant with the functional test suite.
    
    Partial revert of r9605:
    Using DN.toNormalizedString() should have been only done for replication metadata, not for user data because OpenDJ stores and returns data like provided by users.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9709 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

10 Oct, 2013

6 commits

  • Fixed a regression introduced in r9677. It was found by Christophe Sovant with the functional test suite:
    When throwing a DirectoryException with ERR_RESYNC_REQUIRED_UNKNOWN_DOMAIN_IN_PROVIDED_COOKIE, the domainCtxts is read before it is written to.
    This code should have been changed to iterate on the Set<DomainContext> rather than the domainCtxts field which has not been assigned at this point.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9707 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • DBCursor.java:
    Added usage example to the javadocs.
    
    DraftCNDB.java:
    Replaced catch (Exception) by catch (DatabaseException), or removed them altogether where unapplicable.
    In next(), first wiped the current record + reverted the if condition and clauses.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9704 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Changes after review from Matthew Swift.
    
    Renamed CNIndexRecord to ChangeNumberIndexRecord to be consistent with ChangeNumberIndexDB name.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9703 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 
    Removed ChangeNumberIndexDB.getRecord(long).
    
    
    ECLServerHandler.java:
    Replaced calls to ChangeNumberIndexDB.getRecord() by calls to ChangeNumberIndexDB.getCursorFrom().
    Inlined all the crossDomainStartState local variables.
    Added getCursorFrom() to preserve old behaviour from JEChangeNumberIndexDBCursor ctor.
    
    
    ChangeNumberIndexDB.java:
    Removed getRecord().
    
    JEChangeNumberIndexDB.java:
    Removed getReadCursor(), getRecord() + some surprising and never used code like lock field, hasLock(), lock() and release().
    
    JEChangeNumberIndexDBCursor.java:
    In ctor, do not throw ChangelogException if the changeNumber cannot be found.
    
    JEChangeNumberIndexDBTest.java:
    Used JEChangeNumberIndexDB.getCursorFrom() instead of JEChangeNumberIndexDB.getReadCursor().
    Renamed a few variables.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9702 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Changes after review from Matthew Swift.
    
    Consolidated ChangeNumberIndexDBCursor and ReplicaDBCursor into a single interface named DBCursor.
    
    
    
    DBCursor.java: ADDED
    It does not implement Comparable like ReplicaDBCursor. Instead a Comparator has been extracted to JEChangelogDB.CrossReplicaDBCursor .
    
    ChangeNumberIndexDBCursor.java, ReplicaDBCursor.java: DELETED - replaced by DBCursor
    
    *.java:
    Used DBCursor instead of removed cursor interfaces.
    
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9699 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 
    
    Here is the failure scenario for this test (with one CPU):
    
     t1 (test thread)                  t2 (CNIndexDB purge thread)
     ----------------                  ---------------------------
            |                              |
            |                              v
            |                         JEChangeNumberIndexDB
            |                          run()
            |                          trim()
            |                          clear(null) <- entered, but not completed
            v
    ChangeNumberIndexDB
     setPurgeDelay(0);
     addRecord(1)
     addRecord(2)
     addRecord(3)
            |                              |
            |                              +--> executes the clear()
            v
     getFirstRecord() <- BOOM NullPointerException!
    
    
    
    Ensured the CNIndexDB is not trimmed until we explicitly test it:
    to do this, ensured the thread is only started when needed by the test.
    Also took the occasion to ensure a faster shutdown for the CNIndexDB trimming thread.
    
    
    
    JEChangeNumberIndexDB.java:
    Ensured we use volatile and AtomicBoolean.
    Renamed thread field to trimmingThread.
    Extracted method startTrimmingThread().
    In trim(), added an AtomicBoolean parameter.
    Added methods clear(DN, AtomicBoolean shutdown) and mustShutdown() to ensure faster shutdown.
    
    
    
    JEChangelogDB.java:
    Called JEChangeNumberIndexDB.startTrimmingThread() after creating the JEChangeNumberIndexDB instance.
    
    JEChangeNumberIndexDBTest.java:
    Extracted constants.
    In testTrim(), called JEChangeNumberIndexDB.startTrimmingThread().
    In testClear(), removed
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9698 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

09 Oct, 2013

6 commits

  • Improved javadocs.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9696 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 
    Change after review from Matthew Swift.
    
    
    ReplicaDBCursor.java:
    In next(), added thrown ChangelogException.
    
    ReplicaDBCursor.java, JEChangelogDB.java:
    In next(), let ChangelogException go up.
    
    *.java:
    Consequence of the change to ReplicaDBCursor.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9695 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Partially reverted r9691 which was: "In getECLChangeNumberLimits(), brought the database empty code to the top of the method: to get something more readable."
    This code is more complex than I thought.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9693 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 
    ReplicationServerDomain.java:
    Renamed getStartState() to getOldestState().
    
    ServerState.java:
    In duplicateOnlyOlderThan(), changed parameter from CSN to long.
    
    
    ReplicationServer.java:
    Consequence of the change to ServerState and ReplicationServerDomain.
    In getECLChangeNumberLimits(), brought the database empty code to the top of the method: to get something more readable.
    
    ECLServerHandler.java:
    Consequence of the change to ServerState and ReplicationServerDomain.
    Changed a few names / comments.
    
    JEChangeNumberIndexDB.java
    Consequence of the change to ReplicationServerDomain.
    
    
    ExternalChangeLogTest.java
    Consequence of the change to ReplicationServerDomain.
    Code cleanup:
    - removed gblCSN instance field
    - inlined sleep()
    - used connection.process*() + inlined runModifyOperation() and runDeleteOperation().
    - renamed getReplicationDomainStartState() to getDomainOldestState() + added better asserts
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9691 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 
    JEChangeNumberIndexDBTest.java:
    Fixed the build.
    
    ChangeNumberIndexDB.java, JEChangeNumberIndexDB.java:
    Improved the javadocs.
    Made instance variables volatile.
    
    *.java:
    First => Oldest
    Last => Newest
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9690 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Changes after review from Matthew Swift.
    
    ChangeNumberIndexDB.java:
    In addRecord(), added a precision to the javadoc.
    
    ReplicationDomainDB.java
    In getDomainLatestTrimDate(), added a precision to the javadoc.
    Added replicaHeartbeat() and replicaOffline().
    
    JEChangelogDB.java:
    Added empty implementation for replicaHeartbeat() and replicaOffline().
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9689 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

08 Oct, 2013

3 commits

  • 
    Changes after review from Matthew Swift.
    Hid ChangeNumberIndexDB.nextChangeNumber() from client code.
    
    
    ChangeNumberIndexDB.java, JEChangeNumberIndexDB.java:
    Removed nextChangeNumber().
    Changed addRecord() to return the assigned changeNumber.
    
    ECLServerHandler.java, CNIndexRecord.java:
    Consequence of the change above.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9685 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 
    Leveraged the ReplicaDBCursor interface to hide iterations cross replica DBs from client code.
    Net benefit: more coherent code + reduced coupling between changelogDB and ECL code.
    There is an added benefit: searching on cn=changelog now returns changes in order.
    
    The change was implemented by moving code from MessageHandler to JEChangelogDB.
    
    
    MessageHandler.java:
    Removed nextOldestUpdateMsg(), addCursorIfNotEmpty() and collectAllCursorsWithChanges().
    Extracted method isLateQueueBelowThreshold().
    
    ReplicationServerDomain.java
    Removed getServerIds() and getCursorFrom(int, CSN)
    Added getCursorFrom(CSN) and getCursorFrom(ServerState).
    
    ReplicationDomainDB.java:
    Removed getDomainServerIds() and getCursorFrom(DN, int, CSN)
    Added getCursorFrom(DN, CSN) and getCursorFrom(DN, ServerState).
    
    JEChangelogDB.java
    Added inenr class CrossReplicaDBCursor + moved getCursorFrom(DN, int, CSN) here from enclosing type.
    Removed getDomainServerIds().
    Added getCursorFrom(DN, CSN) and getCursorFrom(DN, ServerState).
    
    ReplicationBackend.java:
    In writeChangesAfterCSN(), removed loop on the serverIds + renamed "rsd" to "rsDomain".
    
    JEReplicaDB.java, JEReplicaDBCursor.java:
    Implemented toString().
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9682 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • git-svn-id: https://svn.forgerock.org/opendj/trunk@9681 41b1ffd8-f28e-4786-ab96-9950f0a78031
    ludo