31 Jan, 2014

1 commit


30 Jan, 2014

1 commit


28 Jan, 2014

2 commits

  • 
    replication*.properties:
    Added exception stacktraces to NOTICE_READER_EXCEPTION_53.
    
    ServerReader.java:
    In run(), logged the stacktrace when calling logError(). Removed redundant call to logException() which logged to debug logger.
    
    
    Session.java:
    Extracted method read().
    
    
    StaticUtils.java:
    In stackTraceToSingleLineString(), when this is not a debug build, added the exception type at the start of the message.
    
    StaticUtilsTest.java: ADDED
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10202 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Code review: Matthew Swift
    
    Caused by r10049.
    Problem is down to the initialization sequence:
    1. thread 1 - MultimasterReplication.initializeSynchronizationProvider()
    1.1. it creates the ReplicationServerListener
    1.1.1. the ReplicationServerListener in turn creates the ReplicationServer
    1.1.1.1. the ReplicationServer in turn creates the ChangelogDB
    1.1.1.1.1. the ChangelogDB in turn creates the ChangeNumberIndexer thread and STARTs it
    1.1.1.1.1. the ChangelogDB starts the ChangeNumberIndexer thread
    1.2. it proceeds with creating the LDAPReplicationDomain objects one by one
    2. thread 2 - ChangeNumberIndexer.run()
    2.1. it calls ChangeNumberIndexer.initialize()
    2.1.1. ChangeNumberIndexer.initialize() calls MultimasterReplication.isECLEnabledDomain(baseDN)
    
    Steps 1.2. and 2.1.1. are running concurrently.
    If 2.1.1. is run before 1.2. is completed, In ChangeNumberIndexer.initialize():
    1) MultimasterReplication.isECLEnabledDomain(baseDN) returns false, hence a cursor to the relevant replica DBs is not created
    2) then the call to nextChangeForInsertDBCursor.getRecord() returns null, later throwing a NullPointerException because the ChangeNumberIndexer thread is in an illegal state: it was expecting to find an UpdateMsg with the correct CSN stamped on it.
    
    
    
    MultimasterReplication.java:
    Added State enum + state instance member to tell whether MultimasterReplication is ready for work.
    Removed isRegistered instance member superseded by state instance member.
    In isECLEnabledDomain(), completeSynchronizationProvider() and finalizeSynchronizationProvider(), deal with thread waits.
    
    DomainFakeCfg.java:
    Implemented toString().
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10200 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

13 Jan, 2014

2 commits


09 Jan, 2014

1 commit

  • ReplicationDomain.java:
    Made ieContext private and used an AtomicReference for it + In acquireIEContext() and releaseIEContext(), removed now useless synchronized keyword.
    In acquireIEContext(), returned the created ieContext + changed all client code to assign a variable from the result.
    
    ReplicationDomainTest.java:
    Avoid NPE in case the ieContext is released before the asserts.
    Called ReplicationDomain.getImportExportContext().
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10113 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

08 Jan, 2014

2 commits

  • Reduced members' visibilities in LDAPReplicationDomain + removed several methods.
    
    
    ReplicationDomain.java:
    Do not access the IEContext field several times in one method in case it gets updated in the middle, instead accessed it once at the top of a method and passed it down private method calls.
    Moved importInProgress(), getTotalEntryCount(), getLeftEntryCount() to IEContext class.
    Added getImportExportContext().
    
    LDAPReplicationDomain.java:
    Reduced visibility of many methods.
    Removed followImport field + setFollowImport() which can be calculated from importErrorMessageId + added and used isFollowImport() instead.
    Replaced calls to "retrievesBackend(getBaseDN())" with getBackend() + inlined retrievesBackend().
    Called getImportExportContext() to access the IEContext.
    
    FractionalLDIFImportPlugin.java:
    Consequence of removing LDAPReplicationDomain.setFollowImport().
    In doLDIFImport(), factorized code between branches of an if statement + extracted method isNotEmpty().
    In flushFractionalConfigIntoEntry(), used StaticUtils.collectionToString() + extracted method add()
    
    ReplicationMonitor.java:
    In getMonitorData(), simplified the code.
    
    ReplicationDomainTest.java:
    Extracted methods waitEndExport(), assertExportSucessful(), buildExportedData().
    
    *.java:
    Updated copyright years for r10098.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10103 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Changed a lot of subclasses due to this change.
    
    
    ReplicationDomain.java:
    Pulled up generationId, config fields, getGenerationID(), readAssuredConfig() and needReconnection() methods from LDAPReplicationDomain.
    Replaced baseDN, serverID, groupId, refUrls, initWindow fields with new config field + encapsulated fields, updated getters, removed setters
    Replaced assured, assuredMode, assuredSdLevel, assuredTimeout fields with new assuredConfig field + encapsulated fields, updated getters, removed setters
    Removed domains field, never used.
    Inlined stopDomain().
    Added generationId to the ctor + implemented getGenerationID() + added setGenerationID().
    Extracted method restartService(), needsAck().
    
    LDAPReplicationDomain.java:
    Pulled up generationId, config fields, getGenerationID(), readAssuredConfig() and needReconnection() methods to ReplicationDomain.
    In ctor, consequence of the change to ReplicationDomain.
    
    DSInfo.java:
    Made it immutable.
    
    StartSessionMsg.java:
    Code cleanup.
    Javadocs.
    
    
    DummyReplicationDomain.java, FractionalReplicationTest.java, FakeReplicationDomain.java, FakeStressReplicationDomain.java:
    Consequence of the changes to ReplicationDomain.
    Pulled up generationId, getGenerationID(), setGeneration() to ReplicationDomain.
    Called ReplicationDomain.getConfig().
    
    TopologyViewTest.java:
    Consequence of the change to ReplicationDomain.getRefUrls().
    Extracted method checkLists().
    Code cleanup.
    
    AssuredReplicationServerTest.java:
    Streamlined and simplified the createFakeReplicationDomain() methods + removed the boolean assured parameter.
    Added getAssuredType(AssuredMode).
    In FakeReplicationDomain, consequence of the changes to ReplicationDomain.
    In checkUpdateAssuredParameters(), changed the assert a bit to match removal of the boolean assured parameter.
    Removed useless calls to assertNotNull() after calling createReplicationServer() (result is never null).
    Replaced newFakeCfg() by newDomainConfig().
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10098 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

07 Jan, 2014

1 commit


03 Jan, 2014

6 commits

  • Problem was due to the CNIndexDB state not being properly cleaned up.
    I chose to rely on ReplicationTestCase.remove(ReplicationServer) to clean up the state of the ChangeNumberIndexDB.
    
    
    JEChangelogDB.java:
    Extracted getChangeNumberIndexDB(boolean) from getChangeNumberIndexDB().
    
    JEChangeNumberIndexDB.java:
    Removed trimDone field.
    In shutdown(), used Thread.join().
    In run(), removed duplicated code that I unfortunately added there in r9881.
    
    JEChangeNumberIndexDBTest.java:
    Replaced newCNIndexDB() with getCNIndexDBNoTrimming() + Removed createCleanDir() + relied on ReplicationTestCase.remove(ReplicationServer) to clean up the state of the ChangeNumberIndexDB.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10079 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Fixed an infinite loop after r10071 (Why did it not fail when ran locally?) + extracted method assertOnlyNewestRecordIsLeft().
    testClean() this test is still failing in CLI mode, or when you add a dependency towards method testTrim(). I haven't fully grasped yet why is that.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10078 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Code cleanup.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10077 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • More code cleanup and encapsulation.
    
    StartECLSessionMsg.java:
    Introduced enums for ECLRequestType and Persistent.
    Completed javadocs from javadocs coming from ECLServerHandler.
    
    ECLServerHandler.java:
    Replaced individual fields by directly storing the StartECLSessionMsg object.
    Changed visibility to private in a few places.
    Replaced isPersistent() by isNonPersistent() and getSearchPhase() by isInitPhaseDone().
    
    ECLServerWriter.java
    Used better encapsulated methods from ECLServerHandler.
    
    ECLSearchOperation.java, SynchronizationMsgTest.java:
    Consequence of the changes to StartECLSessionMsg.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10076 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • … fake LDAP replication domains, so they can participate in the external changelog.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10073 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • … fake LDAP replication domains, so they can participate in the external changelog.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10072 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

02 Jan, 2014

5 commits

  • 
    Never trim the newest (last) record in the ChangeNumberIndexDB from production code.
    This last record contains the last published change number, which is the one that must be read on startup and used to compute the following monotonically increasing change numbers.
    
    
    JEChangeNumberIndexDB.java:
    Improved and fixed javadocs.
    In ctor, factorized code doing twice the same thing. 
    In addRecord(), updated the newestChangeNumber after a newest record is committed.
    In clear(DN, AtomicBoolean), updated the oldestChangeNumber + do not trim the newest record from the DB.
    
    JEChangeNumberIndexDBTest.java:
    Updated testClear() to assert the newest record is never trimmed by production code.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10071 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Updated copyright.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10070 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • …ated a fake domain for it, so it participates in the external changelog.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10069 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • ChangeNumberIndexer.java:
    Removed useless boolean parameter from ensureCursorExists().
    Changed visibility of getPrecedingCSN() to unit test it.
    
    ChangeNumberIndexerTest.java:
    Added tests for ChangeNumberIndexer.getPrecedingCSN().
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10067 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • BoundedWorkQueueStrategy.java:
    Extracted calls to undesired DirectoryServer static methods into overridable methods.
    
    BoundedWorkQueueStrategyTest.java: ADDED
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10066 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

23 Dec, 2013

2 commits

  • ListenerThread was 100% implemented in terms of ReplicationDomain method calls. It should therefore be part of ReplicationDomain.
    
    * inline ListenerThread into ReplicationDomain and use new DirectoryThread lifecycle methods
    * remove shutdown parameter from ReplicationDomain.processUpdate()
    * reduce visibility of ReplicationDomain.receive()
    * inline ReplicationDomain.processUpdateDoneSynchronous().
    
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10054 41b1ffd8-f28e-4786-ab96-9950f0a78031
    matthew
     
  • 
    The medium consistency point was not progressing on DSRS 2 because the ChangeNumberIndexer mediumConsistencyCSN had been polluted by CSN from "cn=admin data".
    Problem is that ChangeNumberIndexer's publishUpdateMsg() was filtering out updates from ECL disabled domains, but not from domains unknown to ECL.
    On start up, update messages can be received from replication domains have not been been configured yet on the current replication server.
    The fix consists in only updating the medium consistency point with update messages or replica heartbeats coming from explicitly enabled replication domains.
    After this fix, changes from ECL disabled domains will still be stored in dedicated replicaDBs, there is no change in this functionality.
    
    
    MultimasterReplication.java
    Renamed isECLDisabledDomain() to isECLEnabledDomain() so it caters better for the 3 states of the domains: enabled, disabled and unknown
    Code cleanup.
    
    ChangeNumberIndexer.java:
    Extracted method isECLEnabledDomain() to use as a seam for unit testing which calls to MultimasterReplication.isECLEnabledDomain().
    
    ChangeNumberIndexerTest.java
    Added one test emptyDBThreeInitialDSsOneIsNotECLEnabledDomain().
    Renamed assertAddedRecords() to assertExternalChangelogContent(), startIndexer() to startCNIndexer(), stopIndexer() to stopCNIndexer() and indexer to cnIndexer.
    In startCNIndexer(), overrode ChangeNumberIndexer.isNotECLEnabledDomain().
    
    AttributeTypeConstants.java:
    Added support for "cn"
    
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10049 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

20 Dec, 2013

3 commits

  • 
    ReplicationTestCase.java
    Removed emptyOldChanges parameter from openReplicationSession() methods because it has no effect whatsoever.
    
    ReplicationServerTest.java:
    Overrode openReplicationSession() to also check the broker is connected.
    
    InitOnLineTest.java:
    Stopped constantly calling DN.decode() and used instance variable instead.
    
    *.java:
    Consequence of the changes to ReplicationTestCase.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10040 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Had to change integrated unit tests to pass a dummy replication domain.
    
    
    ReplicationBroker.java:
    Removed null checks on the domain.
    
    DummyReplicationDomain.java: ADDED
    
    ReplicationTestCase.java:
    Removed the replicationDomain parameter which were always passed as null.
    Could remove one openReplicationSession() method.
    Passed in a DummyReplicationDomain when creating a ReplicationBroker.
    
    
    ExternalChangeLogTest.java, ReplicationServerTest.java, HistoricalCsnOrderingTest.java, StateMachineTest.java:
    Passed in a DummyReplicationDomain when creating a ReplicationBroker.
    Also consequence of removing the replicationDomain parameter in ReplicationTestCase.openReplicationSession().
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10037 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Fixed compile error.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10036 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

19 Dec, 2013

2 commits

  • 
    ReplicationBroker.java
    Moved session field to ConnectedRS.
    Removed field protocolVersion (useless, superseded by ConnectedRS)
    Reworked ConnectedRS class: removed connected field.
    In ReplicationServerInfo, added setServerURL() + internally stored a RSInfo object + changed connectedDSs from List to Set.
    Used 
    Improved javadocs + removed comments redundant with javadocs.
    
    ComputeBestServerTest.java:
    Consequence of the change to ReplicationBroker.ReplicationServerInfo ctor.
    
    UnbindOperationTestCase.java, ReplicationDomain.java:
    Code cleanup.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10030 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • ReplicationServerTest.java:
    In windowProbeTest(), code only expected to receive a WindowMsg in answer to a WindowProbeMsg sent out. The twist is that it only cares about receiving a WindowMsg which means it can discard every other type of messages in the interim, particularly the ones coming from the protocol: MonitorMsgs, HeartbeatMsgs, etc. Used ReplicationTestCase.waitForSpecificMsg() to achieve this.
    
    AssuredReplicationPluginTest.java:
    Code cleanup.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10027 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

17 Dec, 2013

3 commits

  • 
    Code cleanup: removed code that has been obsoleted by having the separate ChangeNumberIndexer thread.
    Matt and I think that this code was necessary because the change number was computed lazily.
    Now that the change number is computed eagerly, there is no need for this obsolete code.
    
    
    Additional potential work that might be done later:
    - Could we remove the counter records from the replicaDBs?
    - Could we remove ECLServerHandler.eligibleCSN()?
    
    
    ReplicationServer.java:
    In getECLChangeNumberLimits():
    - removed the for loop inferring the "eligibleCount", i.e. the number of changes that are in the replicaDBs but not yet in the ChangeNumberIndexDB
    - removed all the parameters which are now useless
    
    ReplicationServerDomain.java, ReplicationDomainDB.java, JEChangelogDB.java, JEReplicaDB.java, ReplicationDB.java:
    Transitively removed all the methods that were called by ReplicationServer.getECLChangeNumberLimits().
    
    ECLServerHandler.java, FirstChangeNumberVirtualAttributeProvider.java, LastChangeNumberVirtualAttributeProvider.java:
    Removed all the code that was getting and passing in the parameters of ReplicationServer.getECLChangeNumberLimits().
    
    ExternalChangeLogTest.java:
    Removed now useless test ECLReplicationServerFullTest10().
    
    JEReplicaDBTest.java:
    Renamed testDbCounts() to testGetOldestNewestCSNs().
    Removed now useless test testGetCountNoCounterRecords()
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10016 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • …ts and replicas shutting down
    
    
    Replica heartbeats are now making the medium consistency point move forward.
    This brings trunk implementation to be on par with OpenDJ 2.6.0.
    
    
    ReplicationServerDomain.java:
    In getEligibleCSN(), used ReplicationDomainDB.getDomainLastAliveCSNs().
    Removed field ctHeartbeatState + methods getChangeTimeHeartbeatState() and storeReceivedCTHeartbeat(), all superseded by ChangeNumberIndexer.
    
    ExternalChangeLogTest.java
    Consequence of removing ReplicationServerDomain.getChangeTimeHeartbeatState().
    Improved the code readability with waitOpResult() + moved calls to waitOpResult() inside searchOnChangelog().
    
    
    ReplicationDomainDB.java, JEChangelogDB.java:
    Added and implemented getDomainLastAliveCSNs().
    In getDomainOldestCSNs() and getDomainNewestCSNs(), improved javadocs.
    
    ChangeNumberIndexer.java:
    Remain lastSeenUpdates field to lastAliveCSNs + improved javadoc.
    Added getDomainLastAliveCSNs().
    
    
    MultiDomainServerState.java
    Added getServerState(DN).
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10008 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 
    Checkpoint commit that:
    - adds i18n
    - improves comments and javadocs
    - improved ServerState class to internally use a lock free implementation
    
    
    replication.properties:
    Added 2 error messages.
    
    ServerState.java:
    Made the implementation use ConcurrentMap + removed all synchronized blocks + significantly change the code in update(), removeCSN() and toString().
    Removed getMaxCSN() (never used).
    
    ServerStateTest.java:
    Added asserts to testRemoveCSN().
    
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10007 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

13 Dec, 2013

2 commits

  • 
    Most of the replace was done by global search replace from Eclipse via regular expressions.
    Individual files had incorrect headers or badly formatted ones, so some of them have been done manually by replacing the content between CDDL HEADER START and CDDL HEADER END.
    Some headers were quoting "trunk/opends/resource/legal-notices/CDDLv1_0.txt" while it should have been "legal-notices/CDDLv1_0.txt".
    
    
    Removed:
    - resource/legal-notices/OpenDS.LICENSE
    
    The following required manual changes:
    - build.xml
    - ext/checkstyle/opendj.sourceheader
    - resource/admin/java-utilities.xsl
    - src/build-tools/org/opends/build/tools/ProcessFilesForPackages.java
    - src/build-tools/windows/*
    - src/pkg/opends-dsml-gateway_proto.py
    - src/pkg/opends_proto.py
    - src/server/org/opends/server/util/ServerConstants.java
    - tests/staf-tests/functional-tests/shared/data/aci/aci_targattrfilter/add_aci18.ldif
    - tests/staf-tests/build.bat
    
    An incorrect header mentionning "exclude" instead of "include" in the copyright header existed for: tests/staf-tests/functional-tests/testcases/backends/backup/**/*
    
    Some files have no copyright headers at all. I did not update them. Here is some of them (more exist):
    - tests/staf-tests/functional-tests/shared/data/core/psearch/**/*.ref
    - tests/staf-tests/functional-tests/shared/data/core/psearch/**/delete.ldif
    
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9982 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • In testGenerateDoc(), added the @Test annotation + ensured the property set is cleared at the end of the test.
    
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9980 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

11 Dec, 2013

1 commit

  • 
    SetPropSubCommandHandler.java:
    Extracted methods parseValue() and getPropertyDefinition().
    
    Argument.java:
    Improved the toString() + javadocs.
    Minor code cleanups.
    
    ArgumentParser.java:
    Removed one parseArguments() method - never used.
    Extracted method indentAndWrap() to match SubCommandArgumentParser.
    Minor code cleanups.
    
    SubCommandArgumentParser.java:
    Removed one parseArguments() method - never used.
    Minor code cleanups.
    
    DsconfigOptionsTestCase.java:
    Added testGenerateDoc().
    Minor code cleanups.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9959 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

09 Dec, 2013

6 commits