Verified
Status Update
No update yet.
Comments
po...@gmail.com
I tested it on different versions. In 3.6 I see the bug was introduced between in 3.6.2. I am going to bisect to find the exact commit.
The problem arises when reading from the SearchingChange which is initialized here [1].
Apparently, there are some duplicates in the cache when reading the cached data [2] in the GitVisibleChangeFilter.getVisibleChanges.
It could be a problem with the cache loader retrieving duplicates changes, when using query provider to populate the cache.
[1]:https://gerrit.googlesource.com/gerrit/+/refs/heads/stable-3.6/java/com/google/gerrit/server/git/SearchingChangeCacheImpl.java#157
[2]:https://gerrit.googlesource.com/gerrit/+/refs/heads/stable-3.6/java/com/google/gerrit/server/permissions/GitVisibleChangeFilter.java#77
The problem arises when reading from the SearchingChange which is initialized here [1].
Apparently, there are some duplicates in the cache when reading the cached data [2] in the GitVisibleChangeFilter.getVisibleChanges.
It could be a problem with the cache loader retrieving duplicates changes, when using query provider to populate the cache.
[1]:
[2]:
po...@gmail.com
In master (542a03c42793339ba3254b8900570afa57f2c825) it looks ok.
fa...@gmail.com
I confirm the issue is in the cache loader [1]. I have added the following logging at line #168:
```
long dedupedIds = cds.stream().map(c -> c.getId().get()).distinct().count();
long nonDedupedIds = cds.stream().count();
logger.atSevere().log("Deduped %d Vs NON-deduped %d, diff: %d", dedupedIds, nonDedupedIds, nonDedupedIds - dedupedIds);
```
I can see the query to load the cache is returning duplicates, which leads to the ImmutableMap to blow up.
[1]:https://gerrit.googlesource.com/gerrit/+/refs/heads/stable-3.6/java/com/google/gerrit/server/git/SearchingChangeCacheImpl.java#157
```
long dedupedIds = cds.stream().map(c -> c.getId().get()).distinct().count();
long nonDedupedIds = cds.stream().count();
logger.atSevere().log("Deduped %d Vs NON-deduped %d, diff: %d", dedupedIds, nonDedupedIds, nonDedupedIds - dedupedIds);
```
I can see the query to load the cache is returning duplicates, which leads to the ImmutableMap to blow up.
[1]:
po...@gmail.com
This is how to reproduce the issue:
1. Install Gerrit 3.6.2
2. Configure the Gatling tests [1]
3. Run the Gatling tests
Expected results:
All tests should pass
Obtained results:
Some "Clone:http://host.docker.internal:8080/load-test " tests are failing.
The issue has been introduced in 3.6.2 from this commit [2].
Cherry-picking this change [3] in 3.6 fixes the issue, however, the root cause of the issue is still unknown.
This problem doesn't appear in previous releases since the GitVisibleChangeFilter class has been introduced in 3.6.
As mentioned in a previous comment GitVisibleChangeFilter queries the indexes load the SearchingChange used by the filter.
[1]:https://github.com/GerritForge/gatling-sbt-gerrit-test/
[2]:https://gerrit-review.googlesource.com/c/gerrit/+/347737
[3]:https://gerrit-review.googlesource.com/c/gerrit/+/377734
1. Install Gerrit 3.6.2
2. Configure the Gatling tests [1]
3. Run the Gatling tests
Expected results:
All tests should pass
Obtained results:
Some "Clone:
The issue has been introduced in 3.6.2 from this commit [2].
Cherry-picking this change [3] in 3.6 fixes the issue, however, the root cause of the issue is still unknown.
This problem doesn't appear in previous releases since the GitVisibleChangeFilter class has been introduced in 3.6.
As mentioned in a previous comment GitVisibleChangeFilter queries the indexes load the SearchingChange used by the filter.
[1]:
[2]:
[3]:
po...@gmail.com
It looks like the offending change is the following [1]. We reverted it and ran the same tests and they are all passing.
[1]:https://gerrit-review.googlesource.com/c/gerrit/+/345634
[1]:
lu...@gmail.com
Good stuff @Fabio, can we make the internal index pagination configurable then?
I believe this pagination mechanism was a way to serve the "unlimited queries" where the indexing backend didn't support them, hence the internal mechanism was using pagination to simulate this. However, when the internal indexing mechanism is Lucene, the pagination isn't necessarily needed.
Bearing in mind that the "external pagination" is always supported, as it was also done before the [1].
I believe this pagination mechanism was a way to serve the "unlimited queries" where the indexing backend didn't support them, hence the internal mechanism was using pagination to simulate this. However, when the internal indexing mechanism is Lucene, the pagination isn't necessarily needed.
Bearing in mind that the "external pagination" is always supported, as it was also done before the [1].
po...@gmail.com
to...@gmail.com
Comment has been deleted.
sa...@gmail.com
Comment has been deleted.
ap...@gmail.com
Comment has been deleted.
ap...@gmail.com
On Thursday, July 6, 2023 at 12:43:27 AM EDT, <buganizer-system@google.com> wrote:
Replying to this email means your email address will be shared with the team that works on this product.
Changed
_______________________________
Reference Info: 287484350 Git clone failing due to 'Multiple entries with same key' error
component: Gerrit Code Review
status: In Progress (Accepted)
reporter: sy...@gmail.com
assignee: po...@gmail.com
cc: sy...@gmail.com
type: Bug
priority: P1
severity: S1
retention: Component default
Generated by Google IssueTracker notification system
You're receiving this email because you have just been removed from any roles on Google IssueTracker
ap...@google.com
Project: gerrit
Branch: stable-3.6
commit a0da72341b8159c5341af711e6b8991ef1aa7875
Author: Diego Zambelli Sessona <diego.sessona@gmail.com>
Date: Mon Jul 10 14:22:29 2023
Add configuration to disable indexes pagination
Add NONE option to disable index backend pagination,
allowing the user deciding between performance
and data consistency.
This option needs to be honoured by the indexing backend
and this change introduces the correct implementation
for Lucene.
We should also create the corresponding change on the
elasticsearch libModule for throwing an IAE to highlight
that the backend doesn't support NONE pagination type.
API pagination is not affected by this change and is
always preserved.
When having pagination, we could encounter some problems
in the result set, when moving from one page to the
next of the index results:
* duplicated entries (when new changes are created)
* missing entries (when changes are removed or set to
private)
* wrong entries status (when changes are switching
their state during the page switching)
Having duplicates or missing results could have side effects,
depending on the use-case of the consumer of the data.
For example, since indexes are used to populate some LoadingCache
(SearchingChangeCache [1]), inconsistent results returned by
the indexing backend may cause the cache loading function
to fail or keep an inconsistent state which doesn't reflect
the underlying data.
For this case, a workaround [2] has been put in place in master
and 3.8, however, it will not work for the missing
or inconsistent entries.
[1]:https://gerrit.googlesource.com/gerrit/+/refs/heads/stable-3.6/java/com/google/gerrit/server/git/SearchingChangeCacheImpl.java#157
[2]:https://gerrit-review.googlesource.com/c/gerrit/+/349995
Bug:
Release-Notes: Introduce NONE pagination type and its support in Lucene indexing backend
Change-Id: I40f87895d9dac951ae30c5562b2ddbf28b34a41a
Co-Authored-With: Fabio Ponciroli <ponch78@gmail.com>
M Documentation/config-gerrit.txt
M java/com/google/gerrit/index/PaginationType.java
M java/com/google/gerrit/index/query/PaginatingSource.java
M java/com/google/gerrit/index/query/QueryProcessor.java
M java/com/google/gerrit/index/testing/AbstractFakeIndex.java
M java/com/google/gerrit/lucene/AbstractLuceneIndex.java
M java/com/google/gerrit/lucene/LuceneChangeIndex.java
M javatests/com/google/gerrit/server/query/change/AbstractQueryChangesTest.java
M javatests/com/google/gerrit/server/query/change/FakeQueryChangesLatestIndexVersionTest.java
M javatests/com/google/gerrit/server/query/change/FakeQueryChangesPreviousIndexVersionTest.java
M javatests/com/google/gerrit/server/query/change/FakeQueryChangesTest.java
M javatests/com/google/gerrit/server/query/change/LuceneQueryChangesLatestIndexVersionTest.java
M javatests/com/google/gerrit/server/query/change/LuceneQueryChangesPreviousIndexVersionTest.java
A javatests/com/google/gerrit/server/query/group/AbstractFakeQueryGroupsTest.java
M javatests/com/google/gerrit/server/query/group/BUILD
M javatests/com/google/gerrit/server/query/group/FakeQueryGroupsTest.java
https://gerrit-review.googlesource.com/377794
Branch: stable-3.6
commit a0da72341b8159c5341af711e6b8991ef1aa7875
Author: Diego Zambelli Sessona <diego.sessona@gmail.com>
Date: Mon Jul 10 14:22:29 2023
Add configuration to disable indexes pagination
Add NONE option to disable index backend pagination,
allowing the user deciding between performance
and data consistency.
This option needs to be honoured by the indexing backend
and this change introduces the correct implementation
for Lucene.
We should also create the corresponding change on the
elasticsearch libModule for throwing an IAE to highlight
that the backend doesn't support NONE pagination type.
API pagination is not affected by this change and is
always preserved.
When having pagination, we could encounter some problems
in the result set, when moving from one page to the
next of the index results:
* duplicated entries (when new changes are created)
* missing entries (when changes are removed or set to
private)
* wrong entries status (when changes are switching
their state during the page switching)
Having duplicates or missing results could have side effects,
depending on the use-case of the consumer of the data.
For example, since indexes are used to populate some LoadingCache
(SearchingChangeCache [1]), inconsistent results returned by
the indexing backend may cause the cache loading function
to fail or keep an inconsistent state which doesn't reflect
the underlying data.
For this case, a workaround [2] has been put in place in master
and 3.8, however, it will not work for the missing
or inconsistent entries.
[1]:
[2]:
Bug:
Release-Notes: Introduce NONE pagination type and its support in Lucene indexing backend
Change-Id: I40f87895d9dac951ae30c5562b2ddbf28b34a41a
Co-Authored-With: Fabio Ponciroli <ponch78@gmail.com>
M Documentation/config-gerrit.txt
M java/com/google/gerrit/index/PaginationType.java
M java/com/google/gerrit/index/query/PaginatingSource.java
M java/com/google/gerrit/index/query/QueryProcessor.java
M java/com/google/gerrit/index/testing/AbstractFakeIndex.java
M java/com/google/gerrit/lucene/AbstractLuceneIndex.java
M java/com/google/gerrit/lucene/LuceneChangeIndex.java
M javatests/com/google/gerrit/server/query/change/AbstractQueryChangesTest.java
M javatests/com/google/gerrit/server/query/change/FakeQueryChangesLatestIndexVersionTest.java
M javatests/com/google/gerrit/server/query/change/FakeQueryChangesPreviousIndexVersionTest.java
M javatests/com/google/gerrit/server/query/change/FakeQueryChangesTest.java
M javatests/com/google/gerrit/server/query/change/LuceneQueryChangesLatestIndexVersionTest.java
M javatests/com/google/gerrit/server/query/change/LuceneQueryChangesPreviousIndexVersionTest.java
A javatests/com/google/gerrit/server/query/group/AbstractFakeQueryGroupsTest.java
M javatests/com/google/gerrit/server/query/group/BUILD
M javatests/com/google/gerrit/server/query/group/FakeQueryGroupsTest.java
lu...@gmail.com
ap...@google.com
Project: gerrit
Branch: stable-3.4
commit b79ed753852e297d1dc133ab94d4fe6b1e5c6d99
Author: Diego Zambelli Sessona <diego.sessona@gmail.com>
Date: Mon Jul 10 14:22:29 2023
Add configuration to disable internal indexes pagination
Add NONE option to disable index backend pagination,
allowing the user deciding between performance
and data consistency.
This option needs to be honoured by the indexing backend
and this change introduces the correct implementation
for Lucene.
We should also create the corresponding change on the
elasticsearch libModule for throwing an IAE to highlight
that the backend doesn't support NONE pagination type.
API pagination is not affected by this change and is
always preserved.
When having pagination, we could encounter some problems
in the result set, when moving from one page to the
next of the index results:
* duplicated entries (when new changes are created)
* missing entries (when changes are removed or set to
private)
* wrong entries status (when changes are switching
their state during the page switching)
Having duplicates or missing results could have side effects,
depending on the use-case of the consumer of the data.
For example, since indexes are used to populate some LoadingCache
(SearchingChangeCache [1]), inconsistent results returned by
the indexing backend may cause the cache loading function
to fail or keep an inconsistent state which doesn't reflect
the underlying data.
For this case, a workaround [2] has been put in place in master
and 3.8, however, it will not work for the missing
or inconsistent entries.
[1]:https://gerrit.googlesource.com/gerrit/+/refs/heads/stable-3.6/java/com/google/gerrit/server/git/SearchingChangeCacheImpl.java#157
[2]:https://gerrit-review.googlesource.com/c/gerrit/+/349995
Bug:
Release-Notes: Introduce NONE pagination type and its support in Lucene indexing backend
Change-Id: I40f87895d9dac951ae30c5562b2ddbf28b34a41a
Co-Authored-With: Fabio Ponciroli <ponch78@gmail.com>
(cherry picked from commit a0da72341b8159c5341af711e6b8991ef1aa7875)
M Documentation/config-gerrit.txt
M java/com/google/gerrit/index/PaginationType.java
M java/com/google/gerrit/index/query/PaginatingSource.java
M java/com/google/gerrit/index/query/QueryProcessor.java
M java/com/google/gerrit/lucene/AbstractLuceneIndex.java
M java/com/google/gerrit/lucene/LuceneChangeIndex.java
M javatests/com/google/gerrit/server/query/change/AbstractQueryChangesTest.java
M javatests/com/google/gerrit/server/query/change/LuceneQueryChangesLatestIndexVersionTest.java
M javatests/com/google/gerrit/server/query/change/LuceneQueryChangesPreviousIndexVersionTest.java
https://gerrit-review.googlesource.com/379354
Branch: stable-3.4
commit b79ed753852e297d1dc133ab94d4fe6b1e5c6d99
Author: Diego Zambelli Sessona <diego.sessona@gmail.com>
Date: Mon Jul 10 14:22:29 2023
Add configuration to disable internal indexes pagination
Add NONE option to disable index backend pagination,
allowing the user deciding between performance
and data consistency.
This option needs to be honoured by the indexing backend
and this change introduces the correct implementation
for Lucene.
We should also create the corresponding change on the
elasticsearch libModule for throwing an IAE to highlight
that the backend doesn't support NONE pagination type.
API pagination is not affected by this change and is
always preserved.
When having pagination, we could encounter some problems
in the result set, when moving from one page to the
next of the index results:
* duplicated entries (when new changes are created)
* missing entries (when changes are removed or set to
private)
* wrong entries status (when changes are switching
their state during the page switching)
Having duplicates or missing results could have side effects,
depending on the use-case of the consumer of the data.
For example, since indexes are used to populate some LoadingCache
(SearchingChangeCache [1]), inconsistent results returned by
the indexing backend may cause the cache loading function
to fail or keep an inconsistent state which doesn't reflect
the underlying data.
For this case, a workaround [2] has been put in place in master
and 3.8, however, it will not work for the missing
or inconsistent entries.
[1]:
[2]:
Bug:
Release-Notes: Introduce NONE pagination type and its support in Lucene indexing backend
Change-Id: I40f87895d9dac951ae30c5562b2ddbf28b34a41a
Co-Authored-With: Fabio Ponciroli <ponch78@gmail.com>
(cherry picked from commit a0da72341b8159c5341af711e6b8991ef1aa7875)
M Documentation/config-gerrit.txt
M java/com/google/gerrit/index/PaginationType.java
M java/com/google/gerrit/index/query/PaginatingSource.java
M java/com/google/gerrit/index/query/QueryProcessor.java
M java/com/google/gerrit/lucene/AbstractLuceneIndex.java
M java/com/google/gerrit/lucene/LuceneChangeIndex.java
M javatests/com/google/gerrit/server/query/change/AbstractQueryChangesTest.java
M javatests/com/google/gerrit/server/query/change/LuceneQueryChangesLatestIndexVersionTest.java
M javatests/com/google/gerrit/server/query/change/LuceneQueryChangesPreviousIndexVersionTest.java
ap...@google.com
Project: aws-gerrit
Branch: master
commit 18abf4557e3c3ac751606e05a49e2516dfd489ca
Author: Antonio Barone <syntonyze@gmail.com>
Date: Tue Aug 29 15:15:41 2023
Set index pagination type to NONE
Change I40f87895d9dac95 was merged from gerrit 3.4 onwards, to to
disable index backend pagination.
This was done as a fix for the change introduced by I2f5a6aad3798, which
might cause inconsistent, duplicates and missing lucene index results.
Applying paginationType NONE to all AWS recipes.
Bug:
Change-Id: I94509dad9e57370d377ddabea5169d1d0a36d9db
M gerrit/etc/gerrit.config.template
https://gerrit-review.googlesource.com/384614
Branch: master
commit 18abf4557e3c3ac751606e05a49e2516dfd489ca
Author: Antonio Barone <syntonyze@gmail.com>
Date: Tue Aug 29 15:15:41 2023
Set index pagination type to NONE
Change I40f87895d9dac95 was merged from gerrit 3.4 onwards, to to
disable index backend pagination.
This was done as a fix for the change introduced by I2f5a6aad3798, which
might cause inconsistent, duplicates and missing lucene index results.
Applying paginationType NONE to all AWS recipes.
Bug:
Change-Id: I94509dad9e57370d377ddabea5169d1d0a36d9db
M gerrit/etc/gerrit.config.template
ap...@google.com
Project: gerrit
Branch: stable-3.9
commit 1f18afc08acac7f2766309715df694eaa7b261ca
Author: Diego Zambelli Sessona <diego.sessona@gmail.com>
Date: Thu Aug 31 14:44:11 2023
Use PaginatingSource only when pagination enabled
PaginatingSource was used also to handle non-paginated
queries, now the filtering has been restored in
AndSource, so that indices queries can directly use
it when pagination.type is set to NONE.
As stated in change 377794 this was not yet done
for the stable branches to limit the moving parts [1].
[1]:https://gerrit-review.googlesource.com/c/gerrit/+/377794/comment/8e9f0669_19585b73/
Bug:
Release-Notes: skip
Change-Id: Iae3bd95b0d8ca751f9c71f989cdf9b1a71964bc1
M java/com/google/gerrit/index/query/AndSource.java
A java/com/google/gerrit/index/query/FilteredSource.java
M java/com/google/gerrit/index/query/PaginatingSource.java
M java/com/google/gerrit/index/query/QueryProcessor.java
M java/com/google/gerrit/server/query/change/AndChangeSource.java
M javatests/com/google/gerrit/index/query/AndSourceTest.java
M javatests/com/google/gerrit/index/query/PredicateTest.java
https://gerrit-review.googlesource.com/383154
Branch: stable-3.9
commit 1f18afc08acac7f2766309715df694eaa7b261ca
Author: Diego Zambelli Sessona <diego.sessona@gmail.com>
Date: Thu Aug 31 14:44:11 2023
Use PaginatingSource only when pagination enabled
PaginatingSource was used also to handle non-paginated
queries, now the filtering has been restored in
AndSource, so that indices queries can directly use
it when pagination.type is set to NONE.
As stated in change 377794 this was not yet done
for the stable branches to limit the moving parts [1].
[1]:
Bug:
Release-Notes: skip
Change-Id: Iae3bd95b0d8ca751f9c71f989cdf9b1a71964bc1
M java/com/google/gerrit/index/query/AndSource.java
A java/com/google/gerrit/index/query/FilteredSource.java
M java/com/google/gerrit/index/query/PaginatingSource.java
M java/com/google/gerrit/index/query/QueryProcessor.java
M java/com/google/gerrit/server/query/change/AndChangeSource.java
M javatests/com/google/gerrit/index/query/AndSourceTest.java
M javatests/com/google/gerrit/index/query/PredicateTest.java
Description
> Clone:
error_log shows that this is due to:
```
2023-06-16T11:48:53.849Z java.lang.IllegalArgumentException: Multiple entries with same key: 170=ChangeData{Change{170 (I9d9479006d7205398b15e5df9040c8af042cd9cd), dest=load-test,refs/heads/branch-fdad2825c2a4-4-CreateChangeCommand-8c36a042-73ac-47e5-8410-12698b802950-user-4, status=n}} and 170=ChangeData{Change{170 (I9d9479006d7205398b15e5df9040c8af042cd9cd), dest=load-test,refs/heads/branch-fdad2825c2a4-4-CreateChangeCommand-8c36a042-73ac-47e5-8410-12698b802950-user-4, status=n}}
at com.google.common.collect.ImmutableMap.conflictException(ImmutableMap.java:211)
at com.google.common.collect.ImmutableMap.checkNoConflict(ImmutableMap.java:205)
at com.google.common.collect.RegularImmutableMap.checkNoConflictInKeyBucket(RegularImmutableMap.java:146)
at com.google.common.collect.RegularImmutableMap.fromEntryArray(RegularImmutableMap.java:109)
at com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:390)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:582)
at com.google.gerrit.server.permissions.GitVisibleChangeFilter.getVisibleChanges(GitVisibleChangeFilter.java:93)
at com.google.gerrit.server.permissions.DefaultRefFilter.lambda$filter$1(DefaultRefFilter.java:141)
at com.google.common.base.Suppliers$NonSerializableMemoizingSupplier.get(Suppliers.java:167)
at com.google.gerrit.server.permissions.DefaultRefFilter.filterRefs(DefaultRefFilter.java:263)
at com.google.gerrit.server.permissions.DefaultRefFilter.filter(DefaultRefFilter.java:149)
at com.google.gerrit.server.permissions.ProjectControl$ForProjectImpl.filter(ProjectControl.java:417)
at com.google.gerrit.server.git.PermissionAwareReadOnlyRefDatabase.getRefsByPrefix(PermissionAwareReadOnlyRefDatabase.java:138)
at com.google.gerrit.server.git.PermissionAwareReadOnlyRefDatabase.getRefs(PermissionAwareReadOnlyRefDatabase.java:191)
at org.eclipse.jgit.transport.UploadPack.getAdvertisedOrDefaultRefs(UploadPack.java:919)
at org.eclipse.jgit.transport.UploadPack.fetchV2(UploadPack.java:1187)
at org.eclipse.jgit.transport.UploadPack.serveOneCommandV2(UploadPack.java:1359)
at org.eclipse.jgit.transport.UploadPack.serviceV2(UploadPack.java:1420)
at org.eclipse.jgit.transport.UploadPack.uploadWithExceptionPropagation(UploadPack.java:871)
at org.eclipse.jgit.http.server.UploadPackServlet.upload(UploadPackServlet.java:192)
at org.eclipse.jgit.http.server.UploadPackServlet.lambda$doPost$0(UploadPackServlet.java:171)
at com.google.gerrit.httpd.GitOverHttpServlet$GerritUploadPackErrorHandler.upload(GitOverHttpServlet.java:493)
at org.eclipse.jgit.http.server.UploadPackServlet.doPost(UploadPackServlet.java:174)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:212)
at com.google.gerrit.httpd.GitOverHttpServlet$UploadFilter.doFilter(GitOverHttpServlet.java:461)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:210)
at org.eclipse.jgit.http.server.UploadPackServlet$Factory.doFilter(UploadPackServlet.java:137)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:210)
at org.eclipse.jgit.http.server.RepositoryFilter.doFilter(RepositoryFilter.java:112)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:210)
at org.eclipse.jgit.http.server.NoCacheFilter.doFilter(NoCacheFilter.java:53)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:210)
at org.eclipse.jgit.http.server.glue.UrlPipeline.service(UrlPipeline.java:189)
at org.eclipse.jgit.http.server.glue.SuffixPipeline.service(SuffixPipeline.java:70)
at org.eclipse.jgit.http.server.glue.MetaFilter.doFilter(MetaFilter.java:151)
at org.eclipse.jgit.http.server.glue.MetaServlet.service(MetaServlet.java:109)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:290)
at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:280)
at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:184)
at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:89)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
at com.google.gerrit.httpd.raw.StaticModule$PolyGerritFilter.doFilter(StaticModule.java:408)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.GetUserFilter.doFilter(GetUserFilter.java:92)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RequireSslFilter.doFilter(RequireSslFilter.java:72)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
```