Commit Graph

2244 Commits

Author SHA1 Message Date
Neil Alexander
7a9a2547b3
Cross-signing storage code (#1959) 2021-08-04 17:31:18 +01:00
kegsay
ed04eed441
Fix sqlite migration issues (#1960)
* Do not store 'null' in the database for empty JSON arrays

This can cause issues, though it should be noted that the majority
of the time this will marshal/unmarshal just fine, see
https://play.golang.org/p/Doe2NZUgv7Q

* bugfix: sqlite migration should handle create events as having no 'before' snapshot

The state snapshot for any given event in the roomserver represents the state _before_
the event. For the create event, this is nothing, so the state snapshot nid should be 0.

In some cases this wasn't happening, resulting in a nice mix of possible options including:
 - A state snapshot without any state blocks `[]` or `null`.
 - A state snapshot with a single state block with a single event, the create event, causing
   a circular loop. This is incorrect as it represents the state before the event, not after.

* Add state key check
2021-08-04 17:08:17 +01:00
Neil Alexander
da101469fa
Minimum version Go 1.15 (#1957) 2021-08-03 11:35:29 +01:00
kegsay
e3df612953
Add tracing to user API (#1948)
Use the trace version in tests so we can just implement the required API functions.
2021-08-03 11:23:25 +01:00
Neil Alexander
d4aa311c74
Update containerd dependency 2021-08-03 11:17:27 +01:00
Neil Alexander
a81c499f65
Update dependencies (#1956) 2021-08-03 11:15:17 +01:00
kegsay
a22ab3eee6
Read the right username field when performing account deactivation (#1954)
`Login` has 2 username fields, and we were always checking the
deprecated one. Instead, check both.
2021-08-02 13:11:23 +01:00
Kegan Dougal
ed4097825b Factor out StatementList to sqlutil and use it in userapi
It helps with the boilerplate.
2021-07-28 18:30:04 +01:00
Neil Alexander
9e4618000e
Alias key backup endpoints onto /unstable, fix key backup bugs (#1947)
* Default /unstable requests to stable endpoints if not overridden specifically with a custom route

* Rewrite URL

* Try something different

* Fix routing manually

* Fix selectLatestVersionSQL

* Don't return 0 if no backup version exists

* Log more useful error

* fix up replace keys check

* Don't enforce uniqueness on e2e_room_keys_versions_idx

Co-authored-by: kegsay <kegan@matrix.org>
2021-07-28 10:25:45 +01:00
Neil Alexander
3e01a88a0c
Update to neilalexander/utp@54ae7b1 2021-07-27 21:34:40 +01:00
kegsay
32bf14a37c
Key Backups (3/3) : Implement querying keys and various bugfixes (#1946)
* Add querying device keys

Makes a bunch of sytests pass

* Apparently only the current version supports uploading keys

* Linting
2021-07-27 19:29:32 +01:00
kegsay
b3754d68fc
Key Backups (2/3) : Add E2E backup key tables (#1945)
* Add PUT key backup endpoints and glue them to PerformKeyBackup

* Add tables for storing backup keys and glue them into the user API

* Don't create tables whilst still WIPing

* writer on sqlite please

* Linting
2021-07-27 17:08:53 +01:00
kegsay
a060df91e2
Use db writer on sqlite account table (#1944) 2021-07-27 12:47:50 +01:00
kegsay
32538640db
Key backups (1/2) : Add E2E session backup metadata tables (#1943)
* Initial key backup paths and userapi API

* Fix unit tests

* Add key backup table

* Glue REST API to database

* Linting

* use writer on sqlite
2021-07-27 12:47:32 +01:00
Neil Alexander
e3679799ea
Version 0.4.1 2021-07-26 12:52:11 +01:00
kegsay
16bf94f239
Not finding the snapshot is not fatal (#1940) 2021-07-26 12:30:44 +01:00
Meenal Trivedi
fa1ec482a7
fix:Inviting to an unsupported room version return M_BAD_JSON instead of Incompatible_Version (#1930)
* fix:Inviting to an unsupported room version return M_BAD_JSON instead of M_UNSUPPORTED_ROOM_VERSION

Signed-off-by: Meenal Trivedi <meenaltrivedi6102@gmail.com>

* fix

Signed-off-by: Meenal Trivedi <meenaltrivedi6102@gmail.com>

* fix

Signed-off-by: Meenal Trivedi <meenaltrivedi6102@gmail.com>

* feat: make requested changes

Signed-off-by: Meenal Trivedi <meenaltrivedi6102@gmail.com>

* Use error typecast from matrix-org/gomatrixserverlib#272

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-07-26 10:41:58 +01:00
Neil Alexander
75d0f009ec
Don't set prev state when it is the same as the event it replaces (#1936) 2021-07-22 16:46:36 +01:00
Neil Alexander
39e8d1cc6f
Track knocking in membership updater (#1935)
* Topologically sort outliers in SendEventWithState

* Knock in membership updater

* Update gomatrixserverlib

* Update gomatrixserverlib

* Get the NID of the knock event properly for the membership updater
2021-07-22 12:26:58 +01:00
Neil Alexander
43ac66e0b4
Update to matrix-org/gomatrixserverlib@b9eb787 2021-07-22 09:46:31 +01:00
Neil Alexander
9008b1c111
Update to matrix-org/gomatrixserverlib@ae88543 2021-07-21 18:46:31 +01:00
Neil Alexander
7650998308
Update to matrix-org/gomatrixserverlib#271 2021-07-21 17:21:15 +01:00
Neil Alexander
c1447a58e5
Various alias fixes (#1934)
* Generate m.room.canonical_alias instead of legacy m.room.aliases

* Add omitempty tags

* Add aliases endpoint to client API

* Check power levels when setting aliases

* Don't return null on /aliases

* Don't return error if the state event fails

* Update sytest-whitelist

* Don't send updated m.room.canonical_alias events

* Don't check PLs after all because for local aliases they are apparently irrelevant

* Fix some bugs

* Allow deleting a local alias with enough PL

* Fix some more bugs

* Update sytest-whitelist

* Fix copyright notices

* Review comments
2021-07-21 16:53:50 +01:00
Neil Alexander
f0f8c7f055
Optimise QueryServerJoinedToRoom (#1933)
* Optimise checking if a server is in a room

* Fix queries

* Fix queries
2021-07-21 13:06:32 +01:00
Neil Alexander
2f5e4f5991
Support initial_state properly in /createRoom (#1932)
* Refactor room creation to allow initial_state

* GMSL types

* Tweaks to alias

* Fix ordering

* Fix bugs

* Fix create content

* Only unmarshal create content if specified

* Review comments @Kegsay
2021-07-21 12:31:46 +01:00
Neil Alexander
c6acb94af4
Update to matrix-org/gomatrixserverlib#270 2021-07-20 15:14:24 +01:00
J. Ryan Stinnett
034f5ae3f1
Add startup testing for Wasm Pinecone build (#1910)
* Only include go-sqlite3 on the relevant binaries

* The driver name is always sqlite3 now

* Update to matrix-org/go-sqlite3-js@e537baa

* Add initial Wasm test harness

* Upgrade go-sqlite3-js

This fixes an error about semicolons in single statements.

* Add browser-like WebSocket API for testing

* Upgrade go-sqlite3-js

This upgrade includes printing panic messages next to stacks.

* Run for all PRs targeting any branch

* Use manual Node caching

* Temporarily run for all pushes

* Use npm ci instead of install

* Use HTTPS auth for repo packages

* Match path style from build.sh

* update utp

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-07-20 12:14:58 +01:00
Neil Alexander
f63068df3b
Only include go-sqlite3 on the relevant binaries (#1900)
* Only include go-sqlite3 on the relevant binaries

* The driver name is always sqlite3 now

* Update to matrix-org/go-sqlite3-js@e537baa
2021-07-20 11:18:14 +01:00
Neil Alexander
0ff078ebd7
Fixed log printing bug (closes #1885)
Signed-off-by: Kilos kilosliu@workly.ai>
2021-07-20 10:49:12 +01:00
S7evinK
da2023ba04
Rename Riot to Element (#1874)
* s/riot/element/g

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* fix formatting

Co-authored-by: kegsay <kegan@matrix.org>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-07-20 09:45:40 +01:00
David Spenler
8d8fe485b4
Fix failing ban tests (#1884)
* Add room membership and powerlevel checks for func SendBan

* Added non-error return to func GetStateEvent when no state events with the specified state key are found

* Add passing tests to whitelist

* Fixed formatting

* Update roomserver/storage/shared/storage.go

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
Co-authored-by: kegsay <kegan@matrix.org>
Co-authored-by: kegsay <kegsay@gmail.com>
2021-07-19 18:33:05 +01:00
S7evinK
5094bc89bf
Set MaxFileSizeBytes <= 0 to unlimited (#1876)
* Revert "Set MaxFileSizeBytes <= 0 to "unlimited" (#1875)"

This reverts commit 9ed0440421.

* Actually allow unlimited upload

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

Co-authored-by: kegsay <kegan@matrix.org>
2021-07-19 17:58:51 +01:00
Neil Alexander
09d3bab838
Metric fixes
Squashed commit of the following:

commit c6eb4d8bbf80320ec2b6d416c77659b0343e5e47
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Mon Jul 19 16:52:57 2021 +0100

    Fix bug

commit d420966d9ac44936728960a8d38602662b58f1c3
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Mon Jul 19 16:46:12 2021 +0100

    Update metric

commit 0ad6e37846e2ebbbd0e33a38274094bd15b8f11b
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Mon Jul 19 16:30:14 2021 +0100

    Fix observe for calculateStateDurations
2021-07-19 17:20:55 +01:00
Neil Alexander
eb2a8e4c0b
Set buckets for dendrite_roomserver_calculate_state_duration_microseconds 2021-07-19 16:07:06 +01:00
Neil Alexander
b20d402f39
dendrite_roomserver_calculate_state_duration_microseconds as histogram rather than summary 2021-07-19 15:34:12 +01:00
Neil Alexander
628e4cc709
Update to matrix-org/gomatrixserverlib#269 2021-07-19 15:09:01 +01:00
kegsay
af64e648d7
Fix failing Complement tests (#1931)
* Check for missing state keys to avoid panicking

* Check for not allowed errors on send_leave

* More logging

* handle send_join errors too

* Additional send_join checks

* s/join/gmsl.json/
2021-07-19 13:15:19 +01:00
S7evinK
b954343d73
Fix DNS CacheLifetime (#1926)
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
2021-07-16 19:42:53 +01:00
kegsay
728061db03
fedsender: try to satisfy all notary key requests from the cache first (#1925)
* fedsender: try to satisfy all notary key requests from the cache first

* Linting
2021-07-16 11:35:42 +01:00
kegsay
c102adaf43
fedsender: add cache tables for notary keys (#1923)
* Add notary server tables for postgres

* Add sqlite tables

* fedsender: GetServerKeys -> QueryServerKeys

As it now checks a cache and can return multiple responses
2021-07-15 17:45:37 +01:00
Neil Alexander
1827dd7c09
Update dendrite-demo-yggdrasil to Yggdrasil 0.4 (#1921)
* Update Yggdrasil P2P demo for Yggdrasil v0.4

* Build fixes
2021-07-15 12:16:44 +01:00
Kegan Dougal
c211fdf581 Add missing sytest to whitelist 2021-07-14 18:13:00 +01:00
kegsay
e8de96b78c
sytests: fix failing PL tests by updating GMSL (#1920) 2021-07-14 15:48:14 +01:00
Neil Alexander
fa1397c21e
Remove unused binaries 2021-07-14 13:46:35 +01:00
kegsay
2713d1935e
AWSY: update list and shuffle groups (#1919)
So it's more accurate.
2021-07-14 10:48:34 +01:00
kegsay
e80098e186
bugfix: retire invites even when we cannot talk to the remote server to make/send_leave (#1918)
* bugfix: retire invites even when we cannot talk to the remote server to make/send_leave

Also modify the leave response in /sync to include a fake event as this is ultimately
what clients (and sytest) will use to determine leave-ness.

* hash the event ID

* Base64 not hex
2021-07-14 10:39:17 +01:00
kegsay
7df3e691f2
Fix failing complement test (#1917)
Specifically `TestBannedUserCannotSendJoin`
2021-07-13 12:22:27 +01:00
kegsay
f8ae391a5b
Expose more data when outputting output room events (#1916)
* Add more logging for content fields

* Fix fields
2021-07-13 11:19:21 +01:00
kegsay
f698142073
update whitelist (#1914)
* update whitelist

* newline
2021-07-12 19:42:05 +01:00
Kegan Dougal
93cf1e7590 Actually bump GMSL 2021-07-12 18:58:25 +01:00