Commit Graph

3065 Commits

Author SHA1 Message Date
Romain c64e2a319f
Add projects and project boards in exposed metrics (#17202)
* Add projects and project boards in exposed metrics

* Refactor db.GetEngine

Co-authored-by: delvh <dev.lh@web.de>
2021-10-02 03:32:15 +02:00
Norwin 3bbdce2601
API: add html urls to notification subjects (#17178)
* API: add html urls to notification subjects

* add "Repository"

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-09-30 07:17:39 +03:00
techknowlogick e9a9c20d52
Create pub/priv keypair for federation (#17071)
* add logic for creating pub/priv keypair for federation

* Apply suggestions from code review

Co-authored-by: delvh <dev.lh@web.de>

* make fmt

* Update modules/activitypub/keypair.go

Co-authored-by: delvh <dev.lh@web.de>

* add tests

* fix revert

* more tests

* Apply suggestions from code review

Co-authored-by: delvh <dev.lh@web.de>

* make fmt

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: 6543 <6543@obermui.de>
2021-09-28 15:19:22 -04:00
Lunny Xiao 37b29319aa
Fix bug of get context user (#17169)
Co-authored-by: 6543 <6543@obermui.de>
2021-09-28 21:13:04 +08:00
techknowlogick babab0bf5e
Add nodeinfo endpoint for federation purposes (#16953)
Nodeinfo is a way to expose certain metadata about a server for use of discovery regarding functionality of its federation capabilities.

Two endpoints are required:
1. `/.well-known/nodeinfo` which informs client where it can find the location of the location of its metadata (including which version of the schema is used)
2. the endpoint which exposes the metadata in json format according to schema.

Notes:
* `openRegistrations` is a required field, but I propose to set to false as default in case someone writes a crawler to discover "open" gitea instances
* to limit data leakage I also propose to not include the `usage` field (note it is required so it should be included, but left as empty).

More info:
https://github.com/jhass/nodeinfo
https://github.com/jhass/nodeinfo/tree/main/schemas/2.1
http://nodeinfo.diaspora.software/protocol.html
2021-09-28 01:38:06 +02:00
Norwin f48dce3176
Don't return binary file changes in raw PR diffs by default (#17158)
* return diffs without binary file content change

* ?binary=true option to restore old behaviour

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: zeripath <art27@cantab.net>
2021-09-27 17:09:49 -04:00
zeripath e8574f2f7d
Nicely handle missing user in collaborations (#17049)
* Nicely handle missing user in collaborations

It is possible to have a collaboration in a repository which refers to a no-longer
existing user. This causes the repository transfer to fail with an unusual error.

This PR makes `repo.getCollaborators()` nicely handle the missing user by ghosting
the collaboration but also adds consistency check. It also adds an
Access consistency check.

Fix #17044

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2021-09-27 19:07:19 +01:00
zeripath b5856c4437
Create doctor command to fix repo_units broken by dumps from 1.14.3-1.14.6 (#17136)
There was a serious issue with the `gitea dump` command in 1.14.3-1.14.6 which led to corruption of the `config` field of the `repo_unit` table. 

This PR adds a doctor command to attempt to fix the broken repo_units. Users affected by #16961 should run:

```
gitea doctor --fix --run fix-broken-repo-units
```

Fix #16961

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-27 16:55:12 +01:00
Gwyneth Morgan 4e0cca3f7d
Use light/dark theme based on system preference (#17051)
Add a new default theme `auto`, which will automatically switch between
`gitea` (light) and `arc-green` (dark) themes depending on the user's
operating system settings.

Closes: #8183
2021-09-27 15:47:44 +01:00
zeripath 920608e592
Support direct comparison (git diff a..b) as well merge comparison (a...b) (#16635)
This PR changes the compare page to make the "..." in the between branches a clickable
link. This changes the comparison type from "..." to "..". Similarly it makes the
initial compare icon clickable to switch the head and base branches.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-09-27 13:19:34 +01:00
Alexey 〒erentyev 7e9bd206fd
Fix bundle creation (#17079)
Signed-off-by: Alexey Terentyev <axifnx@gmail.com>

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Gwyneth Morgan <87623694+gwymor@users.noreply.github.com>
Co-authored-by: Gwyneth Morgan <gwymor@tilde.club>
2021-09-26 00:29:25 +03:00
Lunny Xiao 91e21d4fca
Move twofactor to models/login (#17143) 2021-09-25 15:00:12 +02:00
zeripath 623d2dd411
Prevent panic in Org mode HighlightCodeBlock (#17140)
When rendering source in org mode there is a mistake in the highlight code that
causes a panic.

This PR fixes this.

Fix #17139

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-24 14:29:32 +01:00
Lunny Xiao 5842a55b31
Move login related structs and functions to models/login (#17093)
* Move login related structs and functions to models/login

* Fix test

* Fix lint

* Fix lint

* Fix lint of windows

* Fix lint

* Fix test

* Fix test

* Only load necessary fixtures when preparing unit tests envs

* Fix lint

* Fix test

* Fix test

* Fix error log

* Fix error log

* Fix error log

* remove unnecessary change

* fix error log

* merge main branch
2021-09-24 19:32:56 +08:00
zeripath 9302eba971
DBContext is just a Context (#17100)
* DBContext is just a Context

This PR removes some of the specialness from the DBContext and makes it context
This allows us to simplify the GetEngine code to wrap around any context in future
and means that we can change our loadRepo(e Engine) functions to simply take contexts.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix unit tests

Signed-off-by: Andrew Thornton <art27@cantab.net>

* another place that needs to set the initial context

Signed-off-by: Andrew Thornton <art27@cantab.net>

* avoid race

Signed-off-by: Andrew Thornton <art27@cantab.net>

* change attachment error

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-23 23:45:36 +08:00
Eng Zer Jun f2e7d5477f
refactor: move from io/ioutil to io and os package (#17109)
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-09-22 13:38:34 +08:00
wxiaoguang b231d0deab
Ignore Sync errors on pipes when doing `CheckAttributeReader.CheckPath`, fix the hang of `git cat-file` (#17096)
* Ignore Sync errors on pipes when doing `CheckAttributeReader.CheckPath`

* apply env patch

* Drop the Sync and fix a number of issues with the Close function

Signed-off-by: Andrew Thornton <art27@cantab.net>

* add logs for DBIndexer and CheckPath

* Fix some more closing bugs

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add test case for language_stats

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update modules/indexer/stats/db.go

Co-authored-by: Lauris BH <lauris@nix.lv>

Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: 6543 <6543@obermui.de>
2021-09-20 20:46:51 +01:00
Lunny Xiao a4bfef265d
Move db related basic functions to models/db (#17075)
* Move db related basic functions to models/db

* Fix lint

* Fix lint

* Fix test

* Fix lint

* Fix lint

* revert unnecessary change

* Fix test

* Fix wrong replace string

* Use *Context

* Correct committer spelling and fix wrong replaced words

Co-authored-by: zeripath <art27@cantab.net>
2021-09-19 19:49:59 +08:00
silverwind d04e581f09
Switch migration icon to svg (#15954)
Followup on https://github.com/go-gitea/gitea/pull/15952, use SVG for migration icon.

<img width="541" alt="Screen Shot 2021-05-23 at 00 26 12" src="https://user-images.githubusercontent.com/115237/119242417-c1a37600-bb5d-11eb-9f97-a80aa89741ee.png">
<img width="540" alt="Screen Shot 2021-05-23 at 00 25 39" src="https://user-images.githubusercontent.com/115237/119242419-c2d4a300-bb5d-11eb-9792-1e6e4092c9f9.png">
2021-09-18 17:22:51 +01:00
zeripath 25533657f6
Add caller to cat-file batch calls (#17082)
Some people still appear to report unclosed cat-files. This PR simply adds the caller
to the process descriptor for the CatFileBatch and CatFileBatchCheck calls.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-09-17 20:54:15 -04:00
zeripath 27b351aba5
Make LDAP be able to skip local 2FA (#16954)
This PR extends #16594 to allow LDAP to be able to be set to skip local 2FA too. The technique used here would be extensible to PAM and SMTP sources.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-17 12:43:47 +01:00
zeripath 8de44d1995
Clean-up HookPreReceive and restore functionality for pushing non-standard refs (#16705)
* Clean-up HookPreReceive and restore functionality for pushing non-standard refs

There was an inadvertent breaking change in #15629 meaning that notes refs and other
git extension refs will be automatically rejected.

Further following #14295 and #15629 the pre-recieve hook code is untenably long and
too complex.

This PR refactors the hook code and removes the incorrect forced rejection of
non-standard refs.

Fix #16688

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-16 15:34:54 +02:00
Lunny Xiao 45ebcb0d1c
Fix bug of migrate comments which only fetch one page (#17055)
* Fix bug of migrate comments which only fetch one page

* add next page to trace

Co-authored-by: zeripath <art27@cantab.net>
2021-09-15 19:30:19 +08:00
zeripath 976db2a8b7
Do not show issue context popup on external issues (#17050)
The issues pop-up context cannot work for external issues - therefore do not show
these.

Fix #17047

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-15 09:45:27 +01:00
Lunny Xiao e2f0ab3343
Add doctor dbconsistency check for release and attachment (#16978) 2021-09-14 20:41:40 +01:00
zeripath 0a542dd59f
Decrement Fork Num when converting from Fork (#17035)
When converting repositories from forks to normal the root NumFork needs to be
decremented too.

Fix #17026

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-14 18:07:08 +01:00
zeripath 26ef180f46
Correctly rollback in ForkRepository (#17034)
The rollback functionality in
services/repository/repository.go:ForkRepository is incorrect and could
lead to a deadlock as it uses DeleteRepository to delete the rolled-back
repository - a function which creates its own transaction.

This PR adjusts the rollback function to only use RemoveAll as any
database changes will be automatically rolled-back. It also handles
panics and adjusts the Close within WithTx to ensure that if there is a
panic the session will always be closed.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-14 18:16:40 +02:00
Jimmy Praet 3d6cb25e31
Support unprotected file patterns (#16395)
Fixes #16381

Note that changes to unprotected files via the web editor still cannot be pushed directly to the protected branch. I could easily add such support for edits and deletes if needed. But for adding, uploading or renaming unprotected files, it is not trivial.

* Extract & Move GetAffectedFiles to modules/git
2021-09-11 16:21:17 +02:00
6543 77f604a928
Add skip and limit to git.GetTags (#16897)
* Make GetTags() api similar to GetBranches()
* Use it for Tag/Release page
2021-09-10 19:30:37 +02:00
6543 51578d6418
Calculate label URL on API (#16186)
close #8028
2021-09-10 18:03:16 +02:00
zeripath 0faf175694
Fix missing close in WalkGitLog (#17008)
When the external context is cancelled it is possible for the
GitLogReader to not itself be Closed.

This PR does three things:

1. Instead of adding a plain defer it wraps the `g.Close` in a func as
`g` may change.
2. It adds the missing explicit g.Close - although the defer fix makes
this unnecessary.
3. It passes down the external context as the base context for the
GitLogReader meaning that the cancellation of the external context will
pass down automatically.

Fix #17007

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-10 17:46:12 +08:00
zeripath 248b96d8a3
Use git attributes to determine generated and vendored status for language stats and diffs (#16773)
Replaces #16262
Replaces #16250
Replaces #14833

This PR first implements a `git check-attr` pipe reader - using `git check-attr --stdin -z --cached` - taking account of the change in the output format in git 1.8.5 and creates a helper function to read a tree into a temporary index file for that pipe reader.

It then wires this in to the language stats helper and into the git diff generation.

Files which are marked generated will be folded by default.

Fixes #14786
Fixes #12653
2021-09-09 21:13:36 +01:00
6543 e14b52ea36
Enable Malayalam, Greek, Persian, Hungarian & Indonesian by default (#16998) 2021-09-09 14:16:42 +08:00
Lunny Xiao efdbba4453
Fix bug of migrated repository not index (#16991)
Fix #16986, #16152
2021-09-08 18:43:19 +01:00
6543 f55cd033ed
Make TestOneDevDownloadRepo work again & more resistant (#16987)
* Update TestOneDevDownloadRepo since OneDev api changed

* Use test functions for migrations
2021-09-08 16:17:22 +02:00
6543 63d7cbcef1
Make mirror feature more configurable (#16957)
Rename`[repository]` `DISABLE_MIRRORS` to `[mirror]` `DISABLE_NEW_PULL`  and add `ENABLED` and `DISABLE_NEW_PUSH` with the below meanings:

- `ENABLED`: **true**: Enables the mirror functionality. Set to **false** to disable all mirrors.
- `DISABLE_NEW_PULL`: **false**: Disable the creation of **new** mirrors. Pre-existing mirrors remain valid.
- `DISABLE_NEW_PUSH`: **false**: Disable the creation of **new** push mirrors. Pre-existing mirrors remain valid.


Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
2021-09-07 17:49:36 +02:00
6543 cf6d398485
Resolve TODO: Enable pagination on GiteaDownloader.GetComments() & update another TODO (#16963)
* Update TODO in migrations

* Resolve TODO: enable pagination on GiteaDownloader.GetComments()
2021-09-06 18:00:50 +02:00
Lunny Xiao a807031a30
Fix storage Iterate bug and Add storage doctor to delete garbage attachments (#16971)
* Fix storage Iterate bug and Add storage doctor to delete garbage attachments

* Close object when used
2021-09-06 16:46:20 +02:00
a1012112796 82da380af7
Add missing icon set for `ActionRenameRepo` (#16972)
Signed-off-by: a1012112796 <1012112796@qq.com>
2021-09-06 16:11:17 +02:00
zeripath 076bb8a2a2
Close storage objects before cleaning (#16934)
Storage.Iterate provides the path and an open object. On windows using
local storage means that the objects will be locked thus preventing clean
from deleting them.

This PR simply closes the objects early.

Fix #16932

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-03 11:48:49 +01:00
zeripath 6c125e9797
Use immediate queues in integration tests and ensure that immediate (#16927)
queue type is also used for unique queues.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-03 11:20:57 +01:00
Lunny Xiao de2e96e37b
Ignore review comment when ref commit is missed (#16905) 2021-09-01 12:33:07 +01:00
6543 7062614dee
Gitlab Migrator: dont ignore reactions of last request (#16903)
Fix bug related to early breaking when migrating reactions.
2021-09-01 11:20:19 +01:00
6543 d21702475b
Fix git.Blob.DataAsync(): close pipe since we return a NopCloser (#16899)
* make sure headGitRepo is closed on err too

* refactor

* Fix git.Blob.DataAsync(): exec cancel since we already read all bytes (close pipe since we return a NopCloser)
2021-08-31 08:43:31 +01:00
zeripath f2b4b0f491
Remove ParseQueueConnStr as it is unused (#16878)
Remove ParseQueueConnStr as `modules/nosql` has taken over all of its functions.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-08-31 05:33:16 +02:00
Lunny Xiao 8d7704b5a2
Fix dump and restore respository (#16698)
* Fix dump and restore

* return different error message for get commit

* Fix missing delete release attachment when deleting repository

* Fix ci and add some comments

Co-authored-by: zeripath <art27@cantab.net>
2021-08-30 21:18:15 -04:00
zeripath 06b9d553bc
Timeout on flush in testing (#16864)
* Timeout on flush in testing

At the end of each test the queues are flushed. At present there is no limit on the
length of time a flush can take which can lead to long flushes.

However, if the CI task is cancelled we lose the log information as to where the long
flush was taking place.

This PR simply adds a default time limit of 2 minutes - at which point an error will
be produced. This should allow us to more easily find the culprit.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* return better error

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2021-08-30 00:27:51 -04:00
Lunny Xiao d985d4bc2f
Paginate releases page & set default page size to 10 (#16857)
* Add release default page and set it to 10

* use limit

Co-authored-by: 6543 <6543@obermui.de>
2021-08-29 18:25:16 +02:00
zeripath f5b0e2c9d2
Simplify split diff view generation and remove JS dependency (#16775)
Gitea has relied on some slow JS code to match up added and deleted lines on the
diff pages. This can cause a considerable slow down on large diff pages.

This PR makes a small change meaning that the matching up can occur much more simply.

Partial fix #1351

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-08-29 15:28:04 +01:00
zeripath c9c0475f4d
In Render tolerate not being passed a context (#16842)
* In Render tolerate not being passed a context

It is possible for RenderString to be passed to an external renderer if markdown
is set to be rendered by an external renderer. No context is currently sent to these
meaning that this will error out.

Fix #16835

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add Context to Repo calls for RenderString

All calls from routers can easily add the context - so add it.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-08-28 21:15:56 +01:00