org.forgerock / org.forgerock.opendj

03 Dec, 2014

1 commit

01 Dec, 2014

1 commit

29 Oct, 2014

2 commits

  • e54f5d39   Updated copyrights after r11216. ... Browse Dir » 
    git-svn-id: https://svn.forgerock.org/opendj/trunk@11218 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 09385456   OPENDJ-1571 Internal Error when doing REST operations on HTTP Connection Handler ... Browse Dir » 
    
As pointed out by Ludo, problem is due to an API change in InternalClientConnection.processSearch() where LinkedHashSets parameters have been replaced with Set.
Reverted revision r9082 which introduced the problem.

Changed all client code to use LinkedHashSets instead of Sets.


CollectionUtils.java:
Added missing methods implemented in TestCaseUtils.

TestCaseUtils.java, LDAPReplicationDomain.java:
Removed generic methods creating collections.

git-svn-id: https://svn.forgerock.org/opendj/trunk@11216 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

28 Oct, 2014

1 commit

23 Oct, 2014

1 commit

07 Oct, 2014

1 commit

22 Sep, 2014

1 commit

17 Sep, 2014

1 commit

  • b0e21f49   Removed unused imports. ... Browse Dir » 
    JEChangeNumberIndexDBTest.java:
Also removed unused field previousCookie.

git-svn-id: https://svn.forgerock.org/opendj/trunk@11014 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

16 Sep, 2014

1 commit

  • bff155b2   OPENDJ-1444 CR-4537 Remove previous cookie from storage of ChangeNumberIndexDB ... Browse Dir » 
    * Implement a new matching strategy for cursors : LESS_THAN_OR_EQUAL_TO_KEY
   for both je and file-based implementations

* Replace the previous cookie by the usage of an ECLMultiDomainDBCursor generated 
   with the medium consistency CSN as start point and the 
   LESS_THAN_OR_EQUAL_TO_KEY strategy
   in classes ChangeNumberIndexer and ChangelogBackend

* Remove storage of the previous cookie in the log for both je and file-based
   implementations

git-svn-id: https://svn.forgerock.org/opendj/trunk@11009 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     

05 Sep, 2014

1 commit

04 Sep, 2014

1 commit

  • 228429b1   Checkpoint commit for OPENDJ-1206 : Create a new ReplicationBackend/ChangelogBackend ... Browse Dir » 
    to support cn=changelog
CR-4439

Implementation of an additional validation of provided cookie when searching changelog
in cookie mode. 

Check for each state of the provided cookie that the csn is not older than the oldest CSN
available in each domain. If this is not the case, then an error 
ERR_RESYNC_REQUIRED_TOO_OLD_DOMAIN_IN_PROVIDED_COOKIE is returned. 

This check was done previously in the ECLServerHandler class but was not ported when
writing ChangelogBackend.  It also means I'm re-adding some methods deleted by the 
previous big clean done by Jean-Noel. 

Changes:
* ReplicationServer.java : 
  - add the check to validateServerState() method
  - refactor validateServerState() method for better readability
* ReplicationServerDomain.java : 
  - re-add getOldestState() method
* ReplicationDomainDB. java : 
  - re-add getDomainOldestCSNs(DN) method
* FileChangelogDB.java, JEChangelogDB.java : 
  - re-add implementation of getDomainOldestCSNs(DN) method


git-svn-id: https://svn.forgerock.org/opendj/trunk@10989 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     

03 Sep, 2014

2 commits

  • 54d4432a   Checkpoint commit for OPENDJ-1206 : Create a new ReplicationBackend/ChangelogBackend ... Browse Dir » 
    to support cn=changelog

Add new test with use of include attributes when searching changelog 

* ChangelogBackendTestCase.java: 
 - implement searchInCookieModeuseOfIncludeAttributes() test
 - minor refactoring


git-svn-id: https://svn.forgerock.org/opendj/trunk@10987 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     
  • e0749381   OPENDJ-1205 (CR-4428) Remove network layer from External ChangeLog implementation ... Browse Dir » 
    After OPENDJ-1206, I could remove all old External ChangeLog (or ECL for short) related classes. I could also remove annoyingly strange APIs from various places of replication code.



ECLUpdateMsg.java, ServerStartECLMsg.java, StartECLSessionMsg.java, ECLServerHandler.java, ECLServerWriter.java, ECLSearchOperation.java, ECLWorkflowElement.java, ExternalChangeLogTest.java: REMOVED
Also removed package src/server/org/opends/server/workflowelement/externalchangelog as part of these deletes.

replication*.properties:
Removed now unused SEVERE_ERR_RESYNC_REQUIRED_TOO_OLD_DOMAIN_IN_PROVIDED_COOKIE.

ReplicationMsg.java:
Deprecated old ECL message types.

ReplicationBroker.java, ReplicationServer.java:
Removed all ECL related code.

ReplicationServerDomain.java:
Removed the now unused otherHandlers field, stopServer(MessageHandler), registerHandler(MessageHandler), unRegisterHandler(MessageHandler), unregisterOtherHandler(), getOldestState() and getLatestDomainTrimDate()

ServerHandler.java
Reduced visibility has much as possible after removing ECLServerHandler.



MultimasterReplication.java:
Renamed getECLDisabledDomains() to , included "cn=changelog" by default + changed client code to not add it anymore.

LastCookieVirtualProvider.java:
Consequence of the change to MultimasterReplication.

ChangelogBackend.java:
Consequence of the change to MultimasterReplication.getECLDisabledDomains().
Removed SearchParams.requestType and replaced its usage with the new isCookieBasedSearch()



MultiDomainServerState.java, ServerState.java, ReplicationDomainDB.java, FileChangelogDB.java, JEChangelogDB.java:
Removed now unused methods / fields.
Made private methods / fields that are now only used inside their declaration classes.



ChangelogBackendTestCase.java, SynchronizationMsgTest.java, FileReplicaDBTest.java:
Updated or removed tests as a consequences of the whole change.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10986 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

01 Sep, 2014

1 commit

  • 8cd57c1b   OPENDJ-1206 (CR-4393) Create a new ReplicationBackend/ChangelogBackend to support cn=changelog ... Browse Dir » 
    Moved persistent searches from workflow elements to the backends.
Made persistent searches work for the ChangelogBackend.
Ensured the base changelog entry is always returned before any changelog entry, even with persistent searches. For this, used SearchOperation attachments to pass information from the "initial search" phase to the "persistent search" phase.
Added ChangelogBackend.notifyEntryAdded() and called it directly from the (JE|File)ChangelogDB and the ChangeNumberIndexer.

FIXME: prevent concurrent execution of initial vs. persistent search phases to avoid sending duplicates.
How do other backends deal with such issue? Apparently they don not deal with it.
TODO: verify changelog read privilege for persistent searches




Backend.java
Added persistentSearches field + registerPersistentSearch() and getPersistentSearches().

ChangelogBackend.java:
Replaced baseChangelogDN with CHANGELOG_BASE_DN and used it throughout + simplified ctor.
Added notifyEntryAdded(), getPersistentSearches(), isPersistentSearch(), isCookieBased(), getNewestCookie(), getChangelogDB(), getInstance(), NumSubordinatesSearchOperation.setAttachment(), TODO JNR
Overrode Backend.registerPersistentSearch().
Extracted method getExcludedDomains().
Renamed SearchParams.multiDomainServerState to cookie.
Removed unused SearchParams.operationId.
Added inner class EntrySender and moved methods to it: matchBaseAndScopeAndFilter(), sendBaseChangelogEntry(), buildBaseChangelogEntry(),
In searchFromCookie(), searchFromChangeNumber() and sendEntryForUpdateMessage(), simplified the code by using EntrySender.
Implemented finalizeBackend().

*Backend.java:
Called super.finalizeBackend()

ChangelogBackendTestCase.java:
Enabled as many tests as possible.



PersistentSearch.java
Added changesOnly field + isChangesOnly() + modified ctor to set it.

ReplicationServer.java:
Replaced old code enabling External Changelog (via workflow element) with new code (with ChangelogBackend).



FileChangelogDB.java, ChangeNumberIndexer.java, JEChangelogDB.java:
Called new ChangelogBackend.notifyEntryAdded().

ChangeNumberIndexerTest.java
Consequence of the change to ChangeNumberIndexer.



LDAPReplicationDomain.java:
Removed old code enabling External Changelog.
Code cleanup.

ExternalChangeLogTest.java:
Disabled tests that do not pass anymore.
Removed old code enabling External Changelog.
Code cleanup.



ECLWorkflowElement.java, LocalBackendWorkflowElement.java:
Removed persistentSearches field + registerPersistentSearch() and getPersistentSearches().

LocalBackendAddOperation.java, LocalBackendDeleteOperation.java, LocalBackendModifyDNOperation.java:
Consequence of moving persistent searches from workflow elements to the backends.

LocalBackendModifyOperation.java:
Consequence of moving persistent searches from workflow elements to the backends.
In performAdditionalPasswordChangedProcessing(), simplified code because we are adding to a Set.
Code cleanup.

LocalBackendSearchOperation.java:
Consequence of moving persistent searches from workflow elements to the backends.
Transformed processSearch field into a local variable.
Code cleanup.

ECLServerHandler.java, ECLServerWriter.java, ECLSearchOperation.java:
Adapted the code to use ChangelogBackend.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10982 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

29 Aug, 2014

1 commit

  • f038ec29   Checkpoint commit for OPENDJ-1206 : Create a new ReplicationBackend/ChangelogBackend ... Browse Dir » 
    to support cn=changelog

* Implement checking of CHANGELOG_READ Privilege
* Implement 2 more tests for changelog backend. 
   Tests are still disabled (until changelog backend is activated in code).

* ChangelogBackend.java: 
 - add new method checkChangelogReadPrivilege() to check CHANGELOG_READ
  privilege
 
* ChangelogBackendTestCase.java: 
 - implement searchInCookieModeAfterDomainIsRemoved test
 - implement searchInCookieModeOnTwoSuffixesWithPrivateBackend test


git-svn-id: https://svn.forgerock.org/opendj/trunk@10979 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     

28 Aug, 2014

1 commit

27 Aug, 2014

1 commit

  • 446d9a53   OPENDJ-1206 (CR-4368) Create a new ReplicationBackend/ChangelogBackend to support cn=changelog ... Browse Dir » 
    Preparation work: ChangeNotificationListener interface only cover a subset of its DirectoryServerPlugin superclass capabilities, so it is redundant.
So I removed ChangeNotificationListener to ease moving persistent searches to the backend.
The two ChangeNotificationListener implementations are now subclasses of InternalDirectoryServerPlugin.
Updated all code after removing ChangeNotificationListener.



ChangeNotificationListener.java: REMOVED



AuthenticatedUsers.java:
Now extends InternalDirectoryServerPlugin instead of implementing ChangeNotificationListener.
Extracted method getNewAuthNDN() + toDN().

CryptoManagerSync.java:
Now extends InternalDirectoryServerPlugin instead of implementing ChangeNotificationListener.
Extracted method getNewAuthNDN() + toDN().



DirectoryServer.java:
Removed references to removed type ChangeNotificationListener.
Be careful with when AuthenticatedUsers is initialized.

DirectoryConfig.java, LocalBackend*Operation.java:
Removed references to removed type ChangeNotificationListener.



CollectionUtils.java: ADDED

TestChangeNotificationListener.java:
Now extends InternalDirectoryServerPlugin instead of implementing ChangeNotificationListener.

AddOperationTestCase.java:
Extracted methods newRawAttributes(), byteStrings(), addSuccess(), addDisconnect(), bind(), writeAddRequest() + used CollectionUtils.
Consequence of the changes to TestChangeNotificationListener.

DeleteOperationTestCase.java:
Consequence of the changes to TestChangeNotificationListener.

ModifyOperationTestCase.java:
Extracted methods newModifications(), newRawModifications() + used CollectionUtils.
Consequence of the changes to TestChangeNotificationListener.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10976 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

26 Aug, 2014

1 commit

  • 6e5a1ee8   Removed useless getChangeNumber() and setChangeNumber methods from *Operation interfaces. ... Browse Dir » 
    These methods are useless because setChangeNumber() is never set inside production code, so changeNumber is always equal to -1 for all the update operations. In addition it is very unlikely the ChangeNumberIndexer could have computed the changeNumber before a results are sent to a persistent search.


AddOperation*.java, DeleteOperation*.java, ModifyDNOperation*.java, ModifyOperation*.java:
Removed changeNumber fields + getters and setters.
Code cleanup.

PersistentSearch.java:
In process*() public methods, removed the long changeNumber attributes.
Removed baseDN, scope and filter fields, and retrieve them directly from the SearchOperation.
Extracted methods isInScope(), isInScopeForModify(), isAnyInScopeForModify(), matchesFilter(), anyMatchesFilter(), sendEntry(), createControls().
to factorize duplicated code.

LocalBackend*Operation.java:
Consequence of the changes to PersistentSearch.process*() methods.
Code cleanup.

ECLServerWriter.java:
In publish(), consequence of the change to PersistentSearch.processAdd().
In doIt(), simplified code by extracting takeECLUpdate() method.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10973 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

21 Aug, 2014

1 commit

  • faa93771   Checkpoint commit for OPENDJ-1206 : Create a new ReplicationBackend/ChangelogBackend ... Browse Dir » 
    to support cn=changelog
CR-4083

Implementation of core features of the changelog backend:
* Initialization and finalization of the backend
* Search of the changelog, in cookie mode (with a cookie control) and
  draft compat mode (by change number)

Note that :
* The support for persistent searches is not implemented yet.
* The changelog backend is currently not branched into code, i.e
 the directory server still uses the ECL workflow for "cn=changelog",
 because it is not possible to enable both changelog backend and ECL
 workflow at the same time (waiting for psearches implementation
before branching the new code)

Code changes:
* ChangelogBackend.java: 
   - implementation of search(), hasSubordinates(),  numSubordinates(),
     and getEntryCount() methods 

* ReplicationServer.java:
   - new dependency on ChangelogBackend and ECLEnabledDomainPredicate
   - new constructor with ECLEnabledDomainPredicate argument
   - new methods enableExternalChangeLog() and shutdownExternalChangelog()
     as future replacement of enabledECL() and shutdownECL()
   - new method getDomainDNs(Set<DN>) for retrieval of domain DNs but an 
     excluded set of dns
   - new method validateServerState(MultiDomainServerState, Set<DN>) for 
     checking coherency of given state with the replication server

* ReplicationDomainDB.java: 
   - new method getCursorFrom(MultiDomainServerState, PositionStrategy, Set<DN>)
     that exclude a given set of domain DNs from the cursor obtained

* FileChangelogDB.java, JEChangelogDB.java:
   - implementation of new method 
     getCursorFrom(MultiDomainServerState, PositionStrategy, Set<DN>)

* ECLEnabledDomainPredicate.java, ECLMultiDomainDBCursor.java:
  - update visibility to public in order to use these classes in ChangelogBackend

* ChangelogBackedTestCase.java:
   - test of ChangelogBackend class, built from ExternalChangeLogTest.java,
     with lots of renaming, refactoring, cleaning compared to original class
   - majority of tests are disabled until the changelog backend is branched into
     code (as these tests require a running server)

* MonitorTest.java:
  - update creation of  ReplicationServer class to use a custom 
    ECLEnabledDomainPredicate
  
 * replication.properties:
   - add messages for changelog backend

git-svn-id: https://svn.forgerock.org/opendj/trunk@10970 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     

20 Aug, 2014

1 commit

19 Aug, 2014

3 commits

  • d49f0da4   OPENDJ-1206 (CR-4261) Create a new ReplicationBackend/ChangelogBackend to support cn=changelog ... Browse Dir » 
    Implemented PositionStrategy.ON_MATCHING_KEY for JEChangelogDB.getCursorFrom(DN baseDN, int serverId).



JEChangelogDB.java, JEReplicaDB.java:
Implemented PositionStrategy.ON_MATCHING_KEY.

JEReplicaDBCursor.java:
Added new field positionStrategy.
Simplified code in next() + added gotoNext().

ReplicationDB.java:
In openReadCursor(), added PositionStrategy parameter.
Made createReplicationKey() handle null CSNs.
Inner class ReplServerDBCursor now implement DBCursor + Added getRecord() + Made next(), now return boolean instead of UpdateMsg + Extracted method computeCurrentRecord().
Added new field "UpdateMsg currentRecord".
Added PositionStrategy parameter to ReplServerDBCursor ctor + protected special code for PositionStrategy.AFTER_MATCHING_KEY.


JEReplicaDBTest.java:
Added test cases for new PositionStrategy ON_MATCHING_KEY.
Split assertFoundInOrder() in two: one each for AFTER_MATCHING_KEY and ON_MATCHING_KEY.
In testGenerateCursorFrom(), changed the code to cover all CSNs.
Extracted method assertNextCSN() and shutdown().
Used SoftAssertions.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10964 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • fee89bdd   OPENDJ-1441 (CR-4244) Persistent searches on external changelog do not return ch… ... Browse Dir » 
    …anges for new replicas and new domains

Follow up of r10944 (and r10912 originally).
Update ReplicaCursors with new replica offline information (For example, when replica came online or when there is more up to date replica offline information).
Renamed ReplicaOfflineCursor(Test) to ReplicaCursor(Test)



FileChangelogDB.java, JEChangelogDB.java:
Added replicaCursors Map field.
In getCursorFrom(), registered replicaCursors.
In unregisterCursor(), handled ReplicaCursors.
In notifyReplicaOnline() and notifyReplicaOffline(), called new method updateCursorsWithOfflineCSN().

ReplicaOfflineCursor.java: RENAMED to ReplicaCursor
Removed returnReplicaOfflineMsg field.
Changed replicaOfflineMsg field from ReplicaOfflineMsg to AtomicReference<ReplicaOfflineMsg>.
In next(), rewrote the code to support resetting the replica offline CSN + added isReplicaOfflineMsgOutdated() and setOfflineCSN()
Added replicaID and domainDB fields + called ReplicationDomainDB.unregisterCursor() in close() + added getReplicaID().

ReplicaOfflineCursorTest.java: RENAMED to ReplicaOfflineCursorTest
Consequence of the change to ReplicaCursor.

ChangeNumberIndexer.java:
Removed outdated TODO

git-svn-id: https://svn.forgerock.org/opendj/trunk@10958 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • f7a274cf   OPENDJ-1441 (CR-4303) Persistent searches on external changelog do not return ch… ... Browse Dir » 
    …anges for new replicas and new domains

Fixing a deadlock situation with ExternalChangeLogTest.
When removing domains, the cursors were not properly closed by ChangeNumberIndexer.run() before calling Object.wait().
Relied on the fact that DBCursors are not thread safe so if client code calls MultiDomainDBCursor.removeDomain(), then the cursor is immediately removed.
I did not see the problem before because unit tests on trunk execute by default with file based changelog db.


ChangeNumberIndexer.java:
In run(), simplified the code that removes the domain cursors.

CompositeDBCursor.java:
Removed removeNoLongerNeededCursors() and removedCursorsIterator() + extracted method removeCursor() from it.
Added another removeCursor() which also removes from the exhaustedCursors.

MultiDomainDBCursor.java
Consequence of the change to CompositeDBCursor.
Removed removeDomains field and removedCursorsIterator().
In removeDomain(), now call CompositeDBCursor.removeCursor().

DomainDBCursor.java:
Consequence of the change to CompositeDBCursor.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10955 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

11 Aug, 2014

1 commit

  • d45736a6   CR-4097 Big code cleanup of some storage schemes. ... Browse Dir » 
    PBKDF2PasswordStorageScheme.java, PKCS5S2PasswordStorageScheme.java:
Fixed a bug in PBKF2 + PKCS5S2 encodeOffline() methods which did not know how to deal with a byte[]. The right way was to wrap it in a ByteString like SaltedSHA1PasswordStorageScheme was already doing. BTW these encodeOffline() methods are never called from production code.
Did more code factorization.
Removed dead code in both with plainPlusSalt local variable which was not used for anything.
Fixed a few bugs in PKCS5S2 where locks where not properly taken (or carefully avoided?).
Made both PBKF2 + PKCS5S2 code look very similar now.

PBKDF2PasswordStorageSchemeTestCase.java, PKCS5S2PasswordStorageSchemeTestCase.java, PasswordStorageSchemeTestCase.java, SaltedSHA1PasswordStorageSchemeTestCase.java, SaltedSHA256PasswordStorageSchemeTestCase.java, SaltedSHA384PasswordStorageSchemeTestCase.java, SaltedSHA512PasswordStorageSchemeTestCase.java:
Removed dead code.

CryptPasswordStorageSchemeTestCase.java:
Made it reuse more code from PasswordStorageSchemeTestCase.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10941 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

25 Jul, 2014

3 commits

  • c4e6860d   Fixed regressions introduced in r10929. ... Browse Dir » 
    PBKDF2PasswordStorageSchemeTestCase.java: ADDED

git-svn-id: https://svn.forgerock.org/opendj/trunk@10931 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • c2c75bfb   OPENDJ-1441 (CR-4082) Persistent searches on external changelog do not return ch… ... Browse Dir » 
    …anges for new replicas and new domains

Created ECLMultiDomainDBCursor to only let ECL enabled domains reach the ChangeNumberIndexer.

ChangeNumberIndexer.java:
Wrapped the MultiDomainDBCursor into an ECLMultiDomainDBCursor.

ECLMultiDomainDBCursor.java: ADDED

JEChangelogDB.java, FileChangelogDB.java:
Removed unnecessary calls to MultimasterREplication.isECLEnabledDomain().

git-svn-id: https://svn.forgerock.org/opendj/trunk@10930 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • eeb64667   Code cleanup for password storage schemes. ... Browse Dir » 
    Used AutoRefactor.
Reduced variable scopes.

PBKDF2PasswordStorageScheme.java:
Extracted methods getDigestBytes(), encodeAndMatch(), 
Removed unnecessary use of StringBuilders.
In passwordMatches() and authPasswordMatches(), used String.indexOf().

PKCS5S2PasswordStorageScheme.java:
See general changes.

PKCS5S2PasswordStorageSchemeTestCase.java:
In getTestPasswords(), reused the super method and filtered entries.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10929 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

21 Jul, 2014

2 commits

  • e4fdfa3d   Checkpoint commit for OPENDJ-1206 : Create a new ReplicationBackend/ChangelogBackend ... Browse Dir » 
    to support cn=changelog
CR-4053

Preparatory work for the changelog backend:  add a new behavior to cursors, in order to be
 able to start cursor at a given key instead of starting just after. 
 In order to avoid introducing another boolean in the methods, I created two enums to 
 define the behavior of a cursor.

Note that JE implementation does not yet implement the new behavior.

* DBCursor class : add two enums KeyMatchingStrategy and PositionStrategy
** The new behavior corresponds to PositionStrategy.ON_MATCHING_KEY, which allow 
 to position on the record with the given key (while AFTER_MATCHING_KEY position 
 just after the record with the given key).

* ReplicationDomainDB : add PositionStrategy argument for all methods that returns a cursor

* ReplicationServerDomain : getCursorFrom(DN, ServerState) method calls underlying method
  with PositionStrategy.AFTER_MATCHING_KEY

* FileChangelogDB : add PositionStrategy argument for all methods that returns a cursor

* FileReplicaDB : add PositionStrategy argument to generateCursorFrom(CSN) method

* FileReplicaDBCursor : add PositionStrategy argument to the constructor, implement the
  new behavior when position strategy is ON_MATCHING_KEY

* DomainDBCursor, MultiDomainDBCursor : add PositionStrategy argument to the constructor, 
 pass the strategy to underlying cursors

* Log, LogFile, BlockLogReader : implement the new behavior when position strategy 
  is ON_MATCHING_KEY

* ChangeNumberIndexer : use AFTER_MATCHING_KEY strategy when retrieving the 
  cursor (no behavior change)

* JEChangelogDB : add PositionStrategy argument for all methods that returns a cursor, 
   but getCursorFrom(DN, int, CSN, PositionStrategy) method does NOT implement the
   new ON_MATCHING_KEY strategy.

* Update of tests classes to match method signature but no new tests added for the 
  new behavior (ON_MATCHING_KEY) - to be done later
  

git-svn-id: https://svn.forgerock.org/opendj/trunk@10922 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     
  • 72f2668d   OPENDJ-1441 (CR-4037) Persistent searches on external changelog do not return ch… ... Browse Dir » 
    …anges for new replicas and new domains


Fixed a problem introduced in r10912.
Problem was due to removing domains from the MultiDomainDBCursor:
In CompositeDBCursor.removeNoLongerNeededCursors(), code iterates over cursors and then forget baseDNs to remove only they match with a cursor.
Problem is that it should be the other way around: iterate over the baseDNs to remove and always forget them whether or not a matching cursor is found.



ChangeNumberIndexer.java:
In run(), better handled the removed domains.

CompositeDBCursor.java:
In removeNoLongerNeededCursors(), iterate over the baseDNs to remove, find a cursor and remove it if found, then always forget the baseDN. 
Added abstract method removedCursorsIterator().
Removed isCursorNoLongerNeededFor() and cursorRemoved().

DomainDBCursor.java, MultiDomainDBCursor.java, CompositeDBCursorTest.java:
Consequence of the changes to CompositeDBCursor.



ExternalChangeLogTest.java:
Code cleanup.
Extracted method readMessages() to factorize code.
Added method assertLastCookieDifferentThanLastValue() to loop until last cookie is updated.
Added inner class Results.

FileChangelogDB.java, JEChangelogDB.java:
In getExistingOrNewDomainMap(), only add the new domain if the baseDN is from an ECL enabled domain.

FileReplicaDBCursor.java:
Fixed javadoc.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10918 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

18 Jul, 2014

1 commit

  • 10df020c   OPENDJ-1441 (CR-4037) Persistent searches on external changelog do not return ch… ... Browse Dir » 
    …anges for new replicas and new domains



This commit only fixes the "new replicas" case.

For this, I made the existing CompositeDBCursor abstract and implemented 2 new cursors: DomainDBCursor and MultiDomainDBCursor which iterate on a single replication domain or in a cross domain fashion.

These new cursors are able to react to topology changes like a new domain or a new replica.
However, ECL persistent search cannot take advantage of them right now because their list of ECLServerHandler.domainCtxts is initialized at the start of the search and never updated after.


Please note that I added new methods to ReplicationDomainDB, but several of them are just called by DomainDBCursor, MultiDomainDBCursor or ChangeNumberIndexer which are considered internal classes to the changelog DB. So maybe should we find a way to hide them from code client to the changelogDB.

These changes meant I could remove the awful double way to use the CompositeDBCursor + I could also remove most cursor management from ChangeNumberIndexer. I think I might find a way in a subsequent commit to also get rid of ChangeNumberIndexer.replicasOffline.

Alas the same changes are duplicated in JE + file based changelog. 



DomainDBCursor.java, MultiDomainDBCursor.java: ADDED

ReplicationDomainDB.java:
Added getCursorFrom(MultiDomainServerState startAfterState) and unregisterCursor(DBCursor) called by DomainDBCursor and MultiDomainDBCursor.

FileChangelogDB.java, JEChangelogDB.java:
Added registeredDomainCursors and registeredMultiDomainCursors fields.
In getExistingOrNewDomainMap(), updated MultiDomainDBCursors when a new domain is created.
In getCursorFrom(DN baseDN, int serverId, CSN), created the ReplicaOfflineCSN there from getCursorFrom(DN baseDN, ServerState).
Added newDomainDBCursor().
Reworked newOfflineCSN().
Implemented new methods in ReplicationDomainDB.
In getOrCreateReplicaDB(), updated DomainDBCursors when a new replica is created.
Synchronized the two files to ease diffing them together.

ChangeNumberIndexer.java:
Removed the responsibility to manage cursors from this class.
Removed allCursors, newCursors fields.
In publishUpdateMsg(), initialize(), moveForwardMediumConsistencyPoint() and run() removed code that dealt with creating/opening/recreating/removing cursors.
Removed resetNextChangeForInsertDBCursor(), ensureCursorExists(), removeCursors(), getCursor(), recycleExhaustedCursors(), createNewCursors(), getPrecedingCSN().
Made getPrecedingCSN() public static.
Added logUnexpectedException().


CompositeDBCursor.java:
Now abstract.
Removed ctor.
Removed recycleExhaustedCursors field.
Added incorporateNewCursors, isCursorNoLongerNeededFor(), cursorRemoved() and addCursor() + used them in next().
In next(), extracted recycleExhaustedCursors + used newly added removeNoLongerNeededCursors().
In close(), completed code.

ChangeNumberIndexerTest.java:
Consequence of the changes to ChangeNumberIndexer.
Renamed cursors field to replicaDBCursors.
Added multiDomainCursor, domainDBCursors fields.
Added eclEnabledDomains field to separate it from startCNIndexer().
Changed setup() and addReplica().

CompositeDBCursorTest.java:
Consequence of the change to CompositeDBCursor.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10912 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

16 Jul, 2014

1 commit

15 Jul, 2014

1 commit

10 Jul, 2014

1 commit

08 Jul, 2014

1 commit

07 Jul, 2014

2 commits

  • 4b38c4f4   Fix unit-test breakage due to changes with the isEqualsTo method. ... Browse Dir » 
    git-svn-id: https://svn.forgerock.org/opendj/trunk@10870 41b1ffd8-f28e-4786-ab96-9950f0a78031
    ludo
     
  • fa189ecd   OPENDJ-1453 Replica offline messages should be synced with updates ... Browse Dir » 
    In r10840, the change to JE/FileChangelogDB.getCursorFrom(DN, ServerState) unnecessarily triggers a lot of calls to the underlying DB (JE or file based) to retrieve the ChangelogState.
As an optimization, keeping an in-memory version of the ChangelogState in synch with the on-disk version will help.


ChangelogState.java:
Now thread safe.
Added removeOfflineReplica(), isEqualTo().
Changed domainToServerIds from Map<DN, List<Integer>> to Map<DN, Set<Integer>>.
In getOfflineReplicas(), now return a MultiDomainServerState.

ChangeNumberIndexer.java:
Consequence of the changes to ChangelogState.


ReplicationDbEnv.java:
Added changelogState field, updated at the same time as the on-disk changelogstate DB
Added getChangelogState(), called by client code instead of readChangelogState().
Renamed readChangelogState() to private readOnDiskChangelogState().
Added stateLock field to sync updates to in-memory and on-disk changelog state.

ReplicationEnvironment.java:
Added changelogState field, updated at the same time as the on-disk changelogstate DB
Added getChangelogState(), called by client code instead of readChangelogState().
Renamed readChangelogState() to private readOnDiskChangelogState().
Renamed domainLock field to domainsLock.

replication.properties:
Removed now unused error message.

FileChangelogDB.java, JEChangelogDB.java:
Consequence of the changes to ChangelogState and ReplicationEnvironment/ReplicationDbEnv.


MultiDomainServerState.java, ServerState.java:
Added getSnapshot() for unit tests.

ReplicationEnvironmentTest.java, ReplicationDbEnvTest.java:
Consequence of the changes to ReplicationEnvironment and ChangelogState.
Used the fake server.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10868 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

02 Jul, 2014

3 commits

  • 05d1a6d0   Code cleanup. ... Browse Dir » 
    AssuredReplicationServerTest.java:
Added SafeReadAssertions with sensible defaults to allow for fluent style assertions on of the FakeReplicationDomain stats.
In checkTimeAndMonitoringSafeData(), extracted methods checkAckOccured() and checkTimeOutOccured().
Used assertj SoftAssertions for blocks of asserts.
In checkServerErrors(), used assertj Assertions.assertThat()
Removed checkServerErrorListsAreEqual() superseded by assertj Assertions.assertThat(Map).

git-svn-id: https://svn.forgerock.org/opendj/trunk@10857 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 83426fb1   Code cleanup. ... Browse Dir » 
    AssuredReplicationServerTest.java:
Removed FakeReplicationDomain.everyUpdatesAreOk duplicating FakeReplicationDomain.nWrongReceivedUpdates.
Added assert methods to inner classes + removed unneeded getters.
Replaced assertContainsOnly() with using assertj.

git-svn-id: https://svn.forgerock.org/opendj/trunk@10856 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 7d54f469   Code cleanup: ... Browse Dir » 
    - hid ReplicationDomain internals
- moved test code out of production code
- removed duplicated test code


ReplicationDomain.java:
In ctor, delegate to other ctor.
In startPublishService(), removed ReplicationDomainCfg parameter and get the config from field instead.
Removed getConfig(), not used anymore.
Moved publish(byte[]) to ReplicationDomainTest.

LDAPReplicationDomain.java, AssuredReplicationServerTest.java, FakeStressReplicationDomain.java:
Consequence of the change to ReplicationDomain.startPublishService().


FakeReplicationDomain.java:
Expanded the code to support what FractionalReplicationTest.FakeReplicationDomain was doing.
Consequence of the change to ReplicationDomain.startPublishService().

FractionalReplicationTest.java:
Removed FakeReplicationDomain inner class, replaced by org.opends.server.replication.service.FakeReplicationDomain.
Removed newConfig(), now unused.

ReplicationDomainTest.java:
Moved ReplicationDomain.publish(byte[]) here.
Extracted method publishRepeatedly().

git-svn-id: https://svn.forgerock.org/opendj/trunk@10855 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac