161f145176
* Add NATS JetStream support Update shopify/sarama * Fix addresses * Don't change Addresses in Defaults * Update saramajetstream * Add missing error check Keep typing events for at least one minute * Use all configured NATS addresses * Update saramajetstream * Try setting up with NATS * Make sure NATS uses own persistent directory (TODO: make this configurable) * Update go.mod/go.sum * Jetstream package * Various other refactoring * Build fixes * Config tweaks, make random jetstream storage path for CI * Disable interest policies * Try to sane default on jetstream base path * Try to use in-memory for CI * Restore storage/retention * Update nats.go dependency * Adapt changes to config * Remove unneeded TopicFor * Dep update * Revert "Remove unneeded TopicFor" This reverts commit f5a4e4a339b6f94ec215778dca22204adaa893d1. * Revert changes made to streams * Fix build problems * Update nats-server * Update go.mod/go.sum * Roomserver input API queuing using NATS * Fix topic naming * Prometheus metrics * More refactoring to remove saramajetstream * Add missing topic * Don't try to populate map that doesn't exist * Roomserver output topic * Update go.mod/go.sum * Message acknowledgements * Ack tweaks * Try to resume transaction re-sends * Try to resume transaction re-sends * Update to matrix-org/gomatrixserverlib@91dadfb * Remove internal.PartitionStorer from components that don't consume keychanges * Try to reduce re-allocations a bit in resolveConflictsV2 * Tweak delivery options on RS input * Publish send-to-device messages into correct JetStream subject * Async and sync roomserver input * Update dendrite-config.yaml * Remove roomserver tests for now (they need rewriting) * Remove roomserver test again (was merged back in) * Update documentation * Docker updates * More Docker updates * Update Docker readme again * Fix lint issues * Send final event in `processEvent` synchronously (since this might stop Sytest from being so upset) * Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that * Go 1.16 instead of Go 1.13 for upgrade tests and Complement * Revert "Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that" This reverts commit 368675283fc44501f227639811bdb16dd5deef8c. * Don't report any errors on `/send` to see what fun that creates * Fix panics on closed channel sends * Enforce state key matches sender * Do the same for leave * Various tweaks to make tests happier Squashed commit of the following: commit 13f9028e7a63662759ce7c55504a9d2423058668 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 15:47:14 2022 +0000 Do the same for leave commit e6be7f05c349fafbdddfe818337a17a60c867be1 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 15:33:42 2022 +0000 Enforce state key matches sender commit 85ede6d64bf10ce9b91cdd6d80f87350ee55242f Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 14:07:04 2022 +0000 Fix panics on closed channel sends commit 9755494a98bed62450f8001d8128e40481d27e15 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 13:38:22 2022 +0000 Don't report any errors on `/send` to see what fun that creates commit 3bb4f87b5dd56882febb4db5621db484c8789b7c Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 13:00:26 2022 +0000 Revert "Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that" This reverts commit 368675283fc44501f227639811bdb16dd5deef8c. commit fe2673ed7be9559eaca134424e403a4faca100b0 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 12:09:34 2022 +0000 Go 1.16 instead of Go 1.13 for upgrade tests and Complement commit 368675283fc44501f227639811bdb16dd5deef8c Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 11:51:45 2022 +0000 Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that commit b028dfc08577bcf52e6cb498026e15fa5d46d07c Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 10:29:08 2022 +0000 Send final event in `processEvent` synchronously (since this might stop Sytest from being so upset) * Merge in NATS Server v2.6.6 and nats.go v1.13 into the in-process connection fork * Add `jetstream.WithJetStreamMessage` to make ack/nak-ing less messy, use process context in consumers * Fix consumer component name in federation API * Add comment explaining where streams are defined * Tweaks to roomserver input with comments * Finish that sentence that I apparently forgot to finish in INSTALL.md * Bump version number of config to 2 * Add comments around asynchronous sends to roomserver in processEventWithMissingState * More useful error message when the config version does not match * Set version in generate-config * Fix version in config.Defaults Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
287 lines
7.8 KiB
Go
287 lines
7.8 KiB
Go
// Copyright 2017 Vector Creations Ltd
|
|
//
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
// you may not use this file except in compliance with the License.
|
|
// You may obtain a copy of the License at
|
|
//
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
//
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
// See the License for the specific language governing permissions and
|
|
// limitations under the License.
|
|
|
|
package config
|
|
|
|
import (
|
|
"fmt"
|
|
"testing"
|
|
)
|
|
|
|
func TestLoadConfigRelative(t *testing.T) {
|
|
_, err := loadConfig("/my/config/dir", []byte(testConfig),
|
|
mockReadFile{
|
|
"/my/config/dir/matrix_key.pem": testKey,
|
|
"/my/config/dir/tls_cert.pem": testCert,
|
|
}.readFile,
|
|
false,
|
|
)
|
|
if err != nil {
|
|
t.Error("failed to load config:", err)
|
|
}
|
|
}
|
|
|
|
const testConfig = `
|
|
version: 2
|
|
global:
|
|
server_name: localhost
|
|
private_key: matrix_key.pem
|
|
key_id: ed25519:auto
|
|
key_validity_period: 168h0m0s
|
|
well_known_server_name: "localhost:443"
|
|
trusted_third_party_id_servers:
|
|
- matrix.org
|
|
- vector.im
|
|
kafka:
|
|
addresses:
|
|
- localhost:2181
|
|
topic_prefix: Dendrite
|
|
use_naffka: true
|
|
naffka_database:
|
|
connection_string: file:naffka.db
|
|
max_open_conns: 100
|
|
max_idle_conns: 2
|
|
conn_max_lifetime: -1
|
|
metrics:
|
|
enabled: false
|
|
basic_auth:
|
|
username: metrics
|
|
password: metrics
|
|
app_service_api:
|
|
internal_api:
|
|
listen: http://localhost:7777
|
|
connect: http://localhost:7777
|
|
database:
|
|
connection_string: file:appservice.db
|
|
max_open_conns: 100
|
|
max_idle_conns: 2
|
|
conn_max_lifetime: -1
|
|
config_files: []
|
|
client_api:
|
|
internal_api:
|
|
listen: http://localhost:7771
|
|
connect: http://localhost:7771
|
|
external_api:
|
|
listen: http://[::]:8071
|
|
registration_disabled: false
|
|
registration_shared_secret: ""
|
|
enable_registration_captcha: false
|
|
recaptcha_public_key: ""
|
|
recaptcha_private_key: ""
|
|
recaptcha_bypass_secret: ""
|
|
recaptcha_siteverify_api: ""
|
|
turn:
|
|
turn_user_lifetime: ""
|
|
turn_uris: []
|
|
turn_shared_secret: ""
|
|
turn_username: ""
|
|
turn_password: ""
|
|
current_state_server:
|
|
internal_api:
|
|
listen: http://localhost:7782
|
|
connect: http://localhost:7782
|
|
database:
|
|
connection_string: file:currentstate.db
|
|
max_open_conns: 100
|
|
max_idle_conns: 2
|
|
conn_max_lifetime: -1
|
|
edu_server:
|
|
internal_api:
|
|
listen: http://localhost:7778
|
|
connect: http://localhost:7778
|
|
federation_api:
|
|
internal_api:
|
|
listen: http://localhost:7772
|
|
connect: http://localhost:7772
|
|
external_api:
|
|
listen: http://[::]:8072
|
|
federation_certificates: []
|
|
federation_sender:
|
|
internal_api:
|
|
listen: http://localhost:7775
|
|
connect: http://localhost:7775
|
|
database:
|
|
connection_string: file:federationapi.db
|
|
max_open_conns: 100
|
|
max_idle_conns: 2
|
|
conn_max_lifetime: -1
|
|
send_max_retries: 16
|
|
disable_tls_validation: false
|
|
proxy_outbound:
|
|
enabled: false
|
|
protocol: http
|
|
host: localhost
|
|
port: 8080
|
|
key_server:
|
|
internal_api:
|
|
listen: http://localhost:7779
|
|
connect: http://localhost:7779
|
|
database:
|
|
connection_string: file:keyserver.db
|
|
max_open_conns: 100
|
|
max_idle_conns: 2
|
|
conn_max_lifetime: -1
|
|
media_api:
|
|
internal_api:
|
|
listen: http://localhost:7774
|
|
connect: http://localhost:7774
|
|
external_api:
|
|
listen: http://[::]:8074
|
|
database:
|
|
connection_string: file:mediaapi.db
|
|
max_open_conns: 100
|
|
max_idle_conns: 2
|
|
conn_max_lifetime: -1
|
|
base_path: ./media_store
|
|
max_file_size_bytes: 10485760
|
|
dynamic_thumbnails: false
|
|
max_thumbnail_generators: 10
|
|
thumbnail_sizes:
|
|
- width: 32
|
|
height: 32
|
|
method: crop
|
|
- width: 96
|
|
height: 96
|
|
method: crop
|
|
- width: 640
|
|
height: 480
|
|
method: scale
|
|
room_server:
|
|
internal_api:
|
|
listen: http://localhost:7770
|
|
connect: http://localhost:7770
|
|
database:
|
|
connection_string: file:roomserver.db
|
|
max_open_conns: 100
|
|
max_idle_conns: 2
|
|
conn_max_lifetime: -1
|
|
server_key_api:
|
|
internal_api:
|
|
listen: http://localhost:7780
|
|
connect: http://localhost:7780
|
|
database:
|
|
connection_string: file:serverkeyapi.db
|
|
max_open_conns: 100
|
|
max_idle_conns: 2
|
|
conn_max_lifetime: -1
|
|
key_perspectives:
|
|
- server_name: matrix.org
|
|
keys:
|
|
- key_id: ed25519:auto
|
|
public_key: Noi6WqcDj0QmPxCNQqgezwTlBKrfqehY1u2FyWP9uYw
|
|
- key_id: ed25519:a_RXGa
|
|
public_key: l8Hft5qXKn1vfHrg3p4+W8gELQVo8N13JkluMfmn2sQ
|
|
sync_api:
|
|
internal_api:
|
|
listen: http://localhost:7773
|
|
connect: http://localhost:7773
|
|
database:
|
|
connection_string: file:syncapi.db
|
|
max_open_conns: 100
|
|
max_idle_conns: 2
|
|
conn_max_lifetime: -1
|
|
user_api:
|
|
internal_api:
|
|
listen: http://localhost:7781
|
|
connect: http://localhost:7781
|
|
account_database:
|
|
connection_string: file:userapi_accounts.db
|
|
max_open_conns: 100
|
|
max_idle_conns: 2
|
|
conn_max_lifetime: -1
|
|
device_database:
|
|
connection_string: file:userapi_devices.db
|
|
max_open_conns: 100
|
|
max_idle_conns: 2
|
|
conn_max_lifetime: -1
|
|
tracing:
|
|
enabled: false
|
|
jaeger:
|
|
serviceName: ""
|
|
disabled: false
|
|
rpc_metrics: false
|
|
tags: []
|
|
sampler: null
|
|
reporter: null
|
|
headers: null
|
|
baggage_restrictions: null
|
|
throttler: null
|
|
logging:
|
|
- type: file
|
|
level: info
|
|
params:
|
|
path: /var/log/dendrite
|
|
`
|
|
|
|
type mockReadFile map[string]string
|
|
|
|
func (m mockReadFile) readFile(path string) ([]byte, error) {
|
|
data, ok := m[path]
|
|
if !ok {
|
|
return nil, fmt.Errorf("no such file %q", path)
|
|
}
|
|
return []byte(data), nil
|
|
}
|
|
|
|
func TestReadKey(t *testing.T) {
|
|
keyID, _, err := readKeyPEM("path/to/key", []byte(testKey), true)
|
|
if err != nil {
|
|
t.Error("failed to load private key:", err)
|
|
}
|
|
wantKeyID := testKeyID
|
|
if wantKeyID != string(keyID) {
|
|
t.Errorf("wanted key ID to be %q, got %q", wantKeyID, keyID)
|
|
}
|
|
}
|
|
|
|
const testKeyID = "ed25519:c8NsuQ"
|
|
|
|
const testKey = `
|
|
-----BEGIN MATRIX PRIVATE KEY-----
|
|
Key-ID: ` + testKeyID + `
|
|
7KRZiZ2sTyRR8uqqUjRwczuwRXXkUMYIUHq4Mc3t4bE=
|
|
-----END MATRIX PRIVATE KEY-----
|
|
`
|
|
|
|
const testCert = `
|
|
-----BEGIN CERTIFICATE-----
|
|
MIIE0zCCArugAwIBAgIJAPype3u24LJeMA0GCSqGSIb3DQEBCwUAMAAwHhcNMTcw
|
|
NjEzMTQyODU4WhcNMTgwNjEzMTQyODU4WjAAMIICIjANBgkqhkiG9w0BAQEFAAOC
|
|
Ag8AMIICCgKCAgEA3vNSr7lCh/alxPFqairp/PYohwdsqPvOD7zf7dJCNhy0gbdC
|
|
9/APwIbPAPL9nU+o9ud1ACNCKBCQin/9LnI5vd5pa/Ne+mmRADDLB/BBBoywSJWG
|
|
NSfKJ9n3XY1bjgtqi53uUh+RDdQ7sXudDqCUxiiJZmS7oqK/mp88XXAgCbuXUY29
|
|
GmzbbDz37vntuSxDgUOnJ8uPSvRp5YPKogA3JwW1SyrlLt4Z30CQ6nH3Y2Q5SVfJ
|
|
NIQyMrnwyjA9bCdXezv1cLXoTYn7U9BRyzXTZeXs3y3ldnRfISXN35CU04Az1F8j
|
|
lfj7nXMEqI/qAj/qhxZ8nVBB+rpNOZy9RJko3O+G5Qa/EvzkQYV1rW4TM2Yme88A
|
|
QyJspoV/0bXk6gG987PonK2Uk5djxSULhnGVIqswydyH0Nzb+slRp2bSoWbaNlee
|
|
+6TIeiyTQYc055pCHOp22gtLrC5LQGchksi02St2ZzRHdnlfqCJ8S9sS7x3trzds
|
|
cYueg1sGI+O8szpQ3eUM7OhJOBrx6OlR7+QYnQg1wr/V+JAz1qcyTC1URcwfeqtg
|
|
QjxFdBD9LfCtfK+AO51H9ugtsPJqOh33PmvfvUBEM05OHCA0lNaWJHROGpm4T4cc
|
|
YQI9JQk/0lB7itF1qK5RG74qgKdjkBkfZxi0OqkUgHk6YHtJlKfET8zfrtcCAwEA
|
|
AaNQME4wHQYDVR0OBBYEFGwb0NgH0Zr7Ga23njEJ85Ozf8M9MB8GA1UdIwQYMBaA
|
|
FGwb0NgH0Zr7Ga23njEJ85Ozf8M9MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEL
|
|
BQADggIBAKU3RHXggbq/pLhGinU5q/9QT0TB/0bBnF1wNFkKQC0FrNJ+ZnBNmusy
|
|
oqOn7DEohBCCDxT0kgOC05gLEsGLkSXlVyqCsPFfycCFhtu1QzSRtQNRxB3pW3Wq
|
|
4/RFVYv0PGBjVBKxImQlEmXJWEDwemGKqDQZPtqR/FTHTbJcaT0xQr5+1oG6lawt
|
|
I/2cW6GQ0kYW/Szps8FgNdSNgVqCjjNIzBYbWhRWMx/63qD1ReUbY7/Yw9KKT8nK
|
|
zXERpbTM9k+Pnm0g9Gep+9HJ1dBFJeuTPugKeSeyqg2OJbENw1hxGs/HjBXw7580
|
|
ioiMn/kMj6Tg/f3HCfKrdHHBFQw0/fJW6o17QImYIpPOPzc5RjXBrCJWb34kxqEd
|
|
NQdKgejWiV/LlVsguIF8hVZH2kRzvoyypkVUtSUYGmjvA5UXoORQZfJ+b41llq1B
|
|
GcSF6iaVbAFKnsUyyr1i9uHz/6Muqflphv/SfZxGheIn5u3PnhXrzDagvItjw0NS
|
|
n0Xq64k7fc42HXJpF8CGBkSaIhtlzcruO+vqR80B9r62+D0V7VmHOnP135MT6noU
|
|
8F0JQfEtP+I8NII5jHSF/khzSgP5g80LS9tEc2ILnIHK1StkInAoRQQ+/HsQsgbz
|
|
ANAf5kxmMsM0zlN2hkxl0H6o7wKlBSw3RI3cjfilXiMWRPJrzlc4
|
|
-----END CERTIFICATE-----
|
|
`
|