Commit Graph

350 Commits

Author SHA1 Message Date
luzpaz e0296b6a6d
Fix various documentation, user-facing, and source comment typos (#16367)
* Fix various doc, user-facing, and source comment typos

Found via `codespell -q 3 -S ./options/locale,./vendor -L ba,pullrequest,pullrequests,readby`
2021-07-08 13:38:13 +02:00
Gary Wang b06342f29c
fix: not able to update local created non-urlencoded wiki pages (#16139)
* fix: not able to update local created non-urlencoded wiki pages

* tidy code

* as per suggestion

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Don't replace space to dash for unescaped wiki filename

Co-authored-by: zeripath <art27@cantab.net>

* Remove incorrect comment

* Remove NameToUnescapedFilename()

Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-07-07 19:23:09 -04:00
KN4CK3R aac663e0da
Implemented head_commit for webhooks (#16282)
* Removed Len field.

* Added head_commit webhook field.

* Added comment for returns.
2021-06-29 15:34:03 +02:00
KN4CK3R 9b1b4b5433
Refactor Webhook + Add X-Hub-Signature (#16176)
This PR removes multiple unneeded fields from the `HookTask` struct and adds the two headers `X-Hub-Signature` and `X-Hub-Signature-256`.

## ⚠️ BREAKING ⚠️ 

* The `Secret` field is no longer passed as part of the payload.
* "Breaking" change (or fix?): The webhook history shows the real called url and not the url registered in the webhook (`deliver.go`@129).

Close #16115
Fixes #7788
Fixes #11755

Co-authored-by: zeripath <art27@cantab.net>
2021-06-27 20:21:09 +01:00
Sergey Dryabzhinsky 22a0636544
Add Visible modes function from Organisation to Users too (#16069)
You can limit or hide organisations. This pull make it also posible for users

- new strings to translte
- add checkbox to user profile form
- add checkbox to admin user.edit form
- filter explore page user search
- filter api admin and public user searches
- allow admins view "hidden" users
- add app option DEFAULT_USER_VISIBILITY
- rewrite many files to use Visibility field
- check for teams intersection
- fix context output
- right fake 404 if not visible

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-06-26 20:53:14 +01:00
KN4CK3R e673e42f7e
Fixed issues not updated by commits (#16254)
`UpdateIssuesCommit` may get called with fewer commits because of `FeedMaxCommitNum` and therefore may miss some commands.
2021-06-26 10:13:51 +01:00
ayb 9b33d18899
Added support for gopher URLs. (#14749)
* Added support for gopher URLs.

* Add setting and make this user settable instead

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-06-25 18:38:27 -04:00
Jimmy Praet 06f483d0c4
Append to existing trailers in generated squash commit message (#15980)
* Remove superfluous newline before Co-authored-by trailers

* Append to existing PR description trailer section

If the existing PR description message already contains a trailer section (e.g. Signed-off-by: ),
append to it instead of creating a new trailer section.

* Reuse compiled regexp

* Simplify regex and deal with trailing \n in PR description

* Add tests for CommitMessageTrailersPattern

- add support for Key:Value (no space after colon)
- add support for whitespace "folding"

* Update services/pull/pull_test.go

Co-authored-by: Norwin <noerw@users.noreply.github.com>

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Norwin <noerw@users.noreply.github.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-06-25 13:01:43 -04:00
6543 3ef23d5411
Use gitea logging module for git module (#16243)
remove log() func from gogs times and switch to proper logging

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-06-25 17:54:08 +01:00
KN4CK3R 44b8b07631
Add tag protection (#15629)
* Added tag protection in hook.

* Prevent UI tag creation if protected.

* Added settings page.

* Added tests.

* Added suggestions.

* Moved tests.

* Use individual errors.

* Removed unneeded methods.

* Switched delete selector.

* Changed method names.

* No reason to be unique.

* Allow editing of protected tags.

* Removed unique key from migration.

* Added docs page.

* Changed date.

* Respond with 404 to not found tags.

* Replaced glob with regex pattern.

* Added support for glob and regex pattern.

* Updated documentation.

* Changed white* to allow*.

* Fixed edit button link.

* Added cancel button.

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-06-25 16:28:55 +02:00
Lunny Xiao b223d36195
Rework repository archive (#14723)
* Use storage to store archive files

* Fix backend lint

* Add archiver table on database

* Finish archive download

* Fix test

* Add database migrations

* Add status for archiver

* Fix lint

* Add queue

* Add doctor to check and delete old archives

* Improve archive queue

* Fix tests

* improve archive storage

* Delete repo archives

* Add missing fixture

* fix fixture

* Fix fixture

* Fix test

* Fix archiver cleaning

* Fix bug

* Add docs for repository archive storage

* remove repo-archive configuration

* Fix test

* Fix test

* Fix lint

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-06-23 17:12:38 -04:00
Viktor Yakovchuk 08f4b3f312
Fix 500 Error with branch and tag sharing the same name #15592 (#16040)
* Fix 500 Error with branch and tag sharing the same name #15592

Fixed 500 error while create Pull request when there are more
than one sources (branch, tag) with the same name

Fix #15592

Signed-off-by: Viktor Yakovchuk <viktor@yakovchuk.net>

* fix logging

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: 6543 <6543@obermui.de>
2021-06-23 17:08:26 -04:00
KN4CK3R 383ffcfa34
Small refactoring of modules/private (#15947)
* Use correct variable name.

* doer is never nil here.

* Use status code constants.

* Replaced generic map with concrete struct.

* Fixed windows lint.

* Removed unused method.

* Changed error codes.

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-06-23 15:38:19 -04:00
6543 be81dc8b20
Localize Email Templates (#16200)
close #14822
2021-06-23 14:33:21 +02:00
Jimmy Praet 17030ced75
Improve notifications for WIP draft PR's (#14663)
* #14559 Reduce amount of email notifications for WIP draft PR's

don't notify repo watchers of WIP draft PR's

* #13190 Notification when WIP Pull Request is ready for review

* Send email notification to repo watchers when WIP PR is created

* Send ui notification to repo watchers when WIP PR is created

* send specific email notification when PR is marked ready for review

instead of reusing the CreatePullRequest action

* Fix lint error

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-06-23 00:14:22 -04:00
KN4CK3R 4fcae3d06d
Add tests for all webhooks (#16214)
* Added tests for MS Teams.

* Added tests for Dingtalk.

* Added tests for Telegram.

* Added tests for Feishu.

* Added tests for Discord.

* Added tests for closed issue and pullrequest comment.

* Added tests for Matrix.

* Trim all spaces.

* Added tests for Slack.

* Added JSONPayload tests.

* Added general tests.

* Replaced duplicated code.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-06-20 22:12:19 -04:00
parnic 35742d4af7
Reintroduce squash merge default comment as a config setting (#16134)
* Reinstate most of commit 09304db9a5

* Move the behaviour behind a config setting

* Also fix the initial #12365
2021-06-19 00:08:22 +02:00
KN4CK3R ebf253b841
Add attachments for PR reviews (#16075)
* First step for multiple dropzones per page.

* Allow attachments on review comments.

* Lint.

* Fixed accidental initialize of the review textarea.

* Initialize SimpleMDE textarea.

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-06-14 21:12:33 -04:00
Nils Dralle f374789fe2
Fix private repo permission problem (#16142)
* Change user access permission

* Add string 'transfer_notices_3'

* Add 3rd transfer note to transfer dialog

* Add test

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-06-14 14:30:35 -04:00
KN4CK3R 440039c0cc
Add push to remote mirror repository (#15157)
* Added push mirror model.

* Integrated push mirror into queue.

* Moved methods into own file.

* Added basic implementation.

* Mirror wiki too.

* Removed duplicated method.

* Get url for different remotes.

* Added migration.

* Unified remote url access.

* Add/Remove push mirror remotes.

* Prevent hangs with missing credentials.

* Moved code between files.

* Changed sanitizer interface.

* Added push mirror backend methods.

* Only update the mirror remote.

* Limit refs on push.

* Added UI part.

* Added missing table.

* Delete mirror if repository gets removed.

* Changed signature. Handle object errors.

* Added upload method.

* Added "upload" unit tests.

* Added transfer adapter unit tests.

* Send correct headers.

* Added pushing of LFS objects.

* Added more logging.

* Simpler body handling.

* Process files in batches to reduce HTTP calls.

* Added created timestamp.

* Fixed invalid column name.

* Changed name to prevent xorm auto setting.

* Remove table header im empty.

* Strip exit code from error message.

* Added docs page about mirroring.

* Fixed date.

* Fixed merge errors.

* Moved test to integrations.

* Added push mirror test.

* Added test.
2021-06-14 19:20:43 +02:00
KN4CK3R 2b39357443
Fixed sanitize errors. (#15240)
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-06-13 23:06:52 -04:00
Lunny Xiao 5fef041079
Remove unnecessary goroutine (#16080)
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-06-09 15:52:55 -04:00
Lunny Xiao fb3ffeb18d
Add sso.Group, context.Auth, context.APIAuth to allow auth special routes (#16086)
* Add sso.Group, context.Auth, context.APIAuth to allow auth special routes

* Remove unnecessary check

* Rename sso -> auth

* remove unused method of Auth interface
2021-06-09 19:53:16 +02:00
Lunny Xiao 0909695204
Merge all deleteBranch as one function and also fix bug when delete branch don't close related PRs (#16067)
* Fix bug when delete branch don't close related PRs

* Merge all deletebranch as one method

* Add missed branch.go

* fix comment

Co-authored-by: Lauris BH <lauris@nix.lv>
2021-06-07 22:52:59 +08:00
KN4CK3R 3607f79d78
Fixed assert statements. (#16089) 2021-06-07 07:27:09 +02:00
zeripath 51775f65bc
Make commit info cancelable (#16032)
* Make modules/context.Context a context.Context

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Simplify context calls

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Set the base context for requests to the HammerContext

Signed-off-by: Andrew Thornton <art27@cantab.net>

* pass context into get-last-commit

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Make commit_info cancellable

Signed-off-by: Andrew Thornton <art27@cantab.net>

* use context as context

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2021-06-06 19:44:58 -04:00
KN4CK3R ee5e1c4a88
Rewrite of the LFS server (#15523)
* Restructured code. Moved static checks out of loop.

* Restructured batch api. Add support for individual errors.

* Let router decide if LFS is enabled.

* Renamed methods.

* Return correct status from verify handler.

* Unified media type check in router.

* Changed error code according to spec.

* Moved checks into router.

* Removed invalid v1 api methods.

* Unified methods.

* Display better error messages.

* Added size parameter. Create meta object on upload.

* Use object error on invalid size.

* Skip upload if object exists.

* Moved methods.

* Suppress fields in response.

* Changed error on accept.

* Added tests.

* Use ErrorResponse object.

* Test against message property.

* Add support for the old invalid lfs client.

* Fixed the check because MinIO wraps the error.

* Use individual repositories.

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
2021-06-06 02:59:27 +03:00
zeripath 3183a465d7
Make modules/context.Context a context.Context (#16031)
* Make modules/context.Context a context.Context

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Simplify context calls

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Set the base context for requests to the HammerContext

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-05-31 02:18:11 -04:00
Jimmy Praet d79c8bc302
Don't manipulate input params in email notification (#16011) 2021-05-30 10:38:38 +01:00
a1012112796 b27a9d43a5
add permission check for ``GenerateRepository`` (#15946)
Signed-off-by: a1012112796 <1012112796@qq.com>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-05-27 12:08:46 -04:00
KN4CK3R 77fa7146c6
Add email headers (#15939)
* Added additional email headers.

* Added tests.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-05-22 14:47:16 +08:00
zeripath 17c5c654a5
Prevent double-login for Git HTTP and LFS and simplify login (#15303)
* Prevent double-login for Git HTTP and LFS and simplify login

There are a number of inconsistencies with our current methods for
logging in for git and lfs. The first is that there is a double login
process. This is particularly evident in 1.13 where there are no less
than 4 hash checks for basic authentication due to the previous
IsPasswordSet behaviour.

This duplicated code had individual inconsistencies that were not
helpful and caused confusion.

This PR does the following:

* Remove the specific login code from the git and lfs handlers except
for the lfs special bearer token
* Simplify the meaning of DisableBasicAuthentication to allow Token and
Oauth2 sign-in.
* The removal of the specific code from git and lfs means that these
both now have the same login semantics and can - if not
DisableBasicAuthentication - login from external services. Further it
allows Oauth2 token authentication as per our standard mechanisms.
* The change in the recovery handler prevents the service from
re-attempting to login - primarily because this could easily cause a
further panic and it is wasteful.

* add test

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-05-15 17:32:09 +02:00
zeripath ba526ceffe
Multiple Queue improvements: LevelDB Wait on empty, shutdown empty shadow level queue, reduce goroutines etc (#15693)
* move shutdownfns, terminatefns and hammerfns out of separate goroutines

Coalesce the shutdownfns etc into a list of functions that get run at shutdown
rather then have them run at goroutines blocked on selects.

This may help reduce the background select/poll load in certain
configurations.

* The LevelDB queues can actually wait on empty instead of polling

Slight refactor to cause leveldb queues to wait on empty instead of polling.

* Shutdown the shadow level queue once it is empty

* Remove bytefifo additional goroutine for readToChan as it can just be run in run

* Remove additional removeWorkers goroutine for workers

* Simplify the AtShutdown and AtTerminate functions and add Channel Flusher

* Add shutdown flusher to CUQ

* move persistable channel shutdown stuff to Shutdown Fn

* Ensure that UPCQ has the correct config

* handle shutdown during the flushing

* reduce risk of race between zeroBoost and addWorkers

* prevent double shutdown

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-05-15 16:22:26 +02:00
zeripath d234d37aa8
Restore PAM user autocreation functionality (#15825)
* Restore PAM user autocreation functionality

PAM autoregistration of users currently fails due to email invalidity.
This PR adds a new setting to PAM to allow an email domain to be set
or just sets the email to the noreply address and if that fails falls
back to uuid@localhost

Fix #15702

Signed-off-by: Andrew Thornton <art27@cantab.net>

* As per KN4CKER

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-05-13 23:11:47 +01:00
KN4CK3R 3d7d750a99
Fix individual tests (addition to #15802) (#15818)
* Decouple TestAction_GetRepoLink and TestSizedAvatarLink.

* Load database for TestCheckGPGUserEmail.

* Load database for TestMakeIDsFromAPIAssigneesToAdd.

* Load database for TestGetUserIDsByNames and TestGetMaileableUsersByIDs.

* Load database for TestUser_ToUser.

* Load database for TestRepository_EditWikiPage.

* Include AppSubURL in test.

* Prevent panic with empty slice.

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-05-12 00:13:42 -04:00
Loutro 34b21625c2
Display specific message if diff is not displayed because of too long line (#15611)
* 7184-  message if line too long

* Update options/locale/locale_en-US.ini

Co-authored-by: silverwind <me@silverwind.io>

* add flag on missing cases

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-05-04 13:58:49 +01:00
6543 1cd301796c
Only log Error on getLastCommitStatus error to let pull list still be visible (#15716) 2021-05-04 14:03:02 +02:00
Lunny Xiao 3d5bb3e6a3
fix webhook timeout bug (#15613)
* Also fix the potential problem in httplib
2021-04-25 21:48:12 +03:00
Lunny Xiao 9d99f6ab19
Refactor renders (#15175)
* Refactor renders

* Some performance optimization

* Fix comment

* Transform reader

* Fix csv test

* Fix test

* Fix tests

* Improve optimaziation

* Fix test

* Fix test

* Detect file encoding with reader

* Improve optimaziation

* reduce memory usage

* improve code

* fix build

* Fix test

* Fix for go1.15

* Fix render

* Fix comment

* Fix lint

* Fix test

* Don't use NormalEOF when unnecessary

* revert change on util.go

* Apply suggestions from code review

Co-authored-by: zeripath <art27@cantab.net>

* rename function

* Take NormalEOF back

Co-authored-by: zeripath <art27@cantab.net>
2021-04-19 18:25:08 -04:00
Lunny Xiao 8202dd1311
Performance improvement for list pull requests (#15447) 2021-04-15 19:34:43 +02:00
zeripath 84f5a0bc62
Always set the merge base used to merge the commit (#15352)
The issue is that the TestPatch will reset the PR MergeBase - and it is possible for TestPatch to update the MergeBase whilst a merge is ongoing. The ensuing merge will then complete but it doesn't re-set the MergeBase it used to merge the PR.

Fixes the intermittent error in git test.

Signed-off-by: Andrew Thornton art27@cantab.net
2021-04-10 09:27:29 +01:00
6543 9c4601bdf8
Code Formats, Nits & Unused Func/Var deletions (#15286)
* _ to unused func options

* rm useless brakets

* rm trifial non used models functions

* rm dead code

* rm dead global vars

* fix routers/api/v1/repo/issue.go

* dont overload import module
2021-04-09 09:40:34 +02:00
KN4CK3R c03e488e14
Add LFS Migration and Mirror (#14726)
* Implemented LFS client.

* Implemented scanning for pointer files.

* Implemented downloading of lfs files.

* Moved model-dependent code into services.

* Removed models dependency. Added TryReadPointerFromBuffer.

* Migrated code from service to module.

* Centralised storage creation.

* Removed dependency from models.

* Moved ContentStore into modules.

* Share structs between server and client.

* Moved method to services.

* Implemented lfs download on clone.

* Implemented LFS sync on clone and mirror update.

* Added form fields.

* Updated templates.

* Fixed condition.

* Use alternate endpoint.

* Added missing methods.

* Fixed typo and make linter happy.

* Detached pointer parser from gogit dependency.

* Fixed TestGetLFSRange test.

* Added context to support cancellation.

* Use ReadFull to probably read more data.

* Removed duplicated code from models.

* Moved scan implementation into pointer_scanner_nogogit.

* Changed method name.

* Added comments.

* Added more/specific log/error messages.

* Embedded lfs.Pointer into models.LFSMetaObject.

* Moved code from models to module.

* Moved code from models to module.

* Moved code from models to module.

* Reduced pointer usage.

* Embedded type.

* Use promoted fields.

* Fixed unexpected eof.

* Added unit tests.

* Implemented migration of local file paths.

* Show an error on invalid LFS endpoints.

* Hide settings if not used.

* Added LFS info to mirror struct.

* Fixed comment.

* Check LFS endpoint.

* Manage LFS settings from mirror page.

* Fixed selector.

* Adjusted selector.

* Added more tests.

* Added local filesystem migration test.

* Fixed typo.

* Reset settings.

* Added special windows path handling.

* Added unit test for HTTPClient.

* Added unit test for BasicTransferAdapter.

* Moved into util package.

* Test if LFS endpoint is allowed.

* Added support for git://

* Just use a static placeholder as the displayed url may be invalid.

* Reverted to original code.

* Added "Advanced Settings".

* Updated wording.

* Added discovery info link.

* Implemented suggestion.

* Fixed missing format parameter.

* Added Pointer.IsValid().

* Always remove model on error.

* Added suggestions.

* Use channel instead of array.

* Update routers/repo/migrate.go

* fmt

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: zeripath <art27@cantab.net>
2021-04-08 18:25:57 -04:00
zeripath fa3895ce81
Move modules/forms to services/forms (#15305)
Forms are dependent on models and therefore should be in services.

This PR also removes the old auth. aliasing

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-04-06 20:44:05 +01:00
6543 80d6c6d7de
[refactor] mailer service (#15072)
* Unexport SendUserMail

* Instead of "[]*models.User" or "[]string" lists infent "[]*MailRecipient" for mailer

* adopt

* code format

* TODOs for "i18n"

* clean

* no fallback for lang -> just use english

* lint

* exec testComposeIssueCommentMessage per lang and use only emails

* rm MailRecipient

* Dont reload from users from db if you alredy have in ram

* nits

* minimize diff

Signed-off-by: 6543 <6543@obermui.de>

* localize subjects

* linter ...

* Tr extend

* start tmpl edit ...

* Apply suggestions from code review

* use translation.Locale

* improve mailIssueCommentBatch

Signed-off-by: Andrew Thornton <art27@cantab.net>

* add i18n to datas

Signed-off-by: Andrew Thornton <art27@cantab.net>

* a comment

Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-04-02 11:25:13 +01:00
zeripath a351b22dc0
Prevent NPE in CommentMustAsDiff if no hunk header (#15199)
I do not understand how this can happen or why.

There is an apparent possibility for a comment.Patch to be missing a hunk header
- this should not happen and do not understand how. But it appears to happen on
1.13 at least in some case.

This PR will simply add a new section if the cursection is empty
thus preventing the NPE.

Fix #15198

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-04-01 14:29:14 -04:00
KN4CK3R 0c6137617f
Add Tabular Diff for CSV files (#14661)
Implements request #14320 The rendering of CSV files does match the diff style.

* Moved CSV logic into base package.

* Added method to create a tabular diff.

* Added CSV compare context.

* Added CSV diff template.

* Use new table style in CSV markup.

* Added file size limit for CSV rendering.

* Display CSV parser errors in diff.

* Lazy read single file.

* Lazy read rows for full diff.

* Added unit tests for various CSV changes.
2021-03-29 22:44:28 +02:00
Lunny Xiao 8567cba0d9
Implement delete release attachments and update release attachments' name (#14130)
* Implement delete release attachment

* Add attachments on release edit page

* Fix bug

* Finish del release attachments

* Fix frontend lint

* Fix tests

* Support edit release attachments

* Added tests

* Remove the unnecessary parameter isCreate from UpdateReleaseOrCreatReleaseFromTag

* Rename UpdateReleaseOrCreatReleaseFromTag to UpdateRelease

* Fix middle align
2021-03-23 00:09:51 +08:00
zeripath 46782d53bc
Place wrapper around comment as diff to catch panics (#15085)
There are a few recurrent issues with comment as diff reporting panics that are resistant to fixing due to the fact that the panic occurs in the template render and is swallowed by the template renderer.

This PR just adds some logging to force the panic to properly logged and re-propagates back up to the template renderer so we can actually detect what the issue is.

Signed-off-by: Andrew Thornton art27@cantab.net
2021-03-21 16:59:58 +00:00
6543 11229ca539
IsUserAllowedToUpdate ignore igonre if user is nil (#14885) 2021-03-04 19:27:54 +01:00
a1012112796 a5279b74b6
Make manual merge autodetection optional and add manual merge as merge method (#12543)
* Make auto check manual merge as a chooseable mod and add manual merge way on ui

as title, Before this pr, we use same way with GH to check manually merge.
It good, but in some special cases, misjudgments can occur. and it's hard
to fix this bug. So I add option to allow repo manager block "auto check manual merge"
function, Then it will have same style like gitlab(allow empty pr). and to compensate for
not being able to detect THE PR merge automatically, I added a manual approach.

Signed-off-by: a1012112796 <1012112796@qq.com>

* make swager

* api support

* ping ci

* fix TestPullCreate_EmptyChangesWithCommits

* Apply suggestions from code review

Co-authored-by: zeripath <art27@cantab.net>

* Apply review suggestions and add test

* Apply suggestions from code review

Co-authored-by: zeripath <art27@cantab.net>

* fix build

* test error message

* make fmt

* Fix indentation issues identified by @silverwind

Co-authored-by: silverwind <me@silverwind.io>

* Fix tests and make manually merged disabled error on API the same

Signed-off-by: Andrew Thornton <art27@cantab.net>

* a small nit

* fix wrong commit id error

* fix bug

* simple test

* fix test

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-03-03 22:41:23 -05:00
zeripath f0e15250b9
Migrate to use jsoniter instead of encoding/json (#14841)
* Migrate to use jsoniter

* fix tests

* update gitea.com/go-chi/binding

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
2021-03-01 22:08:10 +01:00
zeripath 59fd641d1f
When Deleting Repository only explicitly close PRs whose base is not this repository (#14823)
When Deleting Repository only explicitly close PRs whose base is not this repository

Fix #14775

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-03-01 18:39:44 +01:00
6543 a4148c0f12
Repository transfer has to be confirmed, if user can not create repo for new owner (#14792)
* make repo as "pending transfer" if on transfer start doer has no right to create repo in new destination

* if new pending transfer ocured, create UI & Mail notifications
2021-03-01 01:47:30 +01:00
a1012112796 83cf1a894e
Create tag on ui (#13467)
Support create single tag directly

support create tag with message from create release ui

Signed-off-by: a1012112796 <1012112796@qq.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
2021-02-28 19:57:45 +00:00
zeripath 3d8b5ad5f3
Fix a couple of CommentAsPatch issues. (#14804)
* CutDiffAroundLine makes the incorrect assumption that `---` and `+++` always represent part of the header of a diff.

This PR adds a flag to its parsing to prevent this problem and adds a streaming parsing technique to CutDiffAroundLine using an io.pipe instead of just sending data to an unbounded buffer.

Fix #14711

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Handle unquoted comment patch files

When making comment patches unfortunately the patch does not always quote the filename
This makes the diff --git header ambiguous again.

This PR finally adds handling for ambiguity in to parse patch

Fix #14812

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add in testing for no error

There is no way currently for CutDiffAroundLine in this test to cause an
error however, it should still be tested.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-02-27 19:46:14 +01:00
a1012112796 7bf61a89d3
fix a small nit (#14769) 2021-02-22 19:54:01 +01:00
zeripath beb2058186
Fix broken spans in diffs (#14678)
Gitea runs diff on highlighted code fragment for each line in order to provide
code highlight diffs. Unfortunately this diff algorithm is not aware that span tags
and entities are atomic and cannot be split.

The current fixup code makes some attempt to fix these broken tags however, it cannot
handle situations where a tag is split over multiple blocks.

This PR provides a more algorithmic fixup mechanism whereby spans and entities are
completely coalesced into their respective blocks.

This may result in a incompletely reduced diff but - it will definitely prevent the
broken entities and spans that are currently possible.

As a result of this fixup several inconsistencies were discovered in our testcases
and these were also fixed.

Fix #14231

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-02-14 15:51:00 +01:00
vnkmpf 487f2ee41c
Whitespace in commits (#14650)
* Add whitespace to commit view

* Add whitespace to /compare/a...b

* Move repeated whitespaceFlags to gitdiff

* Add whitespace for wiki pages
2021-02-13 05:35:43 +01:00
Paweł Bogusławski 7d7007dca7
Added option to disable webhooks (#13176)
* Added option to disable web hooks

This mod introduces DISABLE_WEB_HOOKS parameter in [security] section
of app.ini (by default set to false). If set to true it disables web
hooks feature. Any existing undelivered web hook tasks will be cancelled.
Any existing web hook definitions will be left untouched in db but
its delivery tasks will be ignored.

Author-Change-Id: IB#1105130

* Webhook spelling fixed

Webhook spelling fixed.

Fixes: 07df6614dc84cdd2e9f39c57577fa1062bd70012
Related: https://github.com/go-gitea/gitea/pull/13176#pullrequestreview-510868421
Author-Change-Id: IB#1105174

* Parameter description fixed

Parameter description fixed.

Fixes: 07df6614dc84cdd2e9f39c57577fa1062bd70012
Related: https://github.com/go-gitea/gitea/pull/13176#pullrequestreview-514086107
Author-Change-Id: IB#1105174
2021-02-11 12:34:34 -05:00
a1012112796 ac701637b4
Add dismiss review feature (#12674)
* Add dismiss review feature

refs:
    https://github.blog/2016-10-12-dismissing-reviews-on-pull-requests/
    https://developer.github.com/v3/pulls/reviews/#dismiss-a-review-for-a-pull-request

* change modal ui and error message

* Add unDismissReview api

Signed-off-by: a1012112796 <1012112796@qq.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
2021-02-11 18:32:25 +01:00
6543 0d1444751f
[API] Add pagination to ListBranches (#14524)
* make PaginateUserSlice generic -> PaginateSlice

* Add pagination to ListBranches

* add skip, limit to Repository.GetBranches()

* Move routers/api/v1/utils/utils PaginateSlice -> modules/util/paginate.go

* repo_module.GetBranches paginate

* fix & rename & more logging

* better description

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: a1012112796 <1012112796@qq.com>
2021-02-03 20:06:13 +01:00
6543 a0e424da85
Enhance Ghost comment mitigation Settings (#14392)
* refactor models.DeleteComment and delete related reactions too

* use deleteComment for UserDeleteWithCommentsMaxDays in DeleteUser

* nits

* Use time.Duration as other time settings have

* docs

* Resolve Fixme & fix potential deadlock

* Disabled by Default

* Update Config Value Description

* switch args

* Update models/issue_comment.go

Co-authored-by: zeripath <art27@cantab.net>

Co-authored-by: zeripath <art27@cantab.net>
2021-01-21 21:56:19 -05:00
Jimmy Praet 4d9349123f
label and milestone webhooks on issue/pull creation (#14363) 2021-01-17 16:15:57 +02:00
a1012112796 d989247bb0
fix mailIssueCommentBatch for pull request (#14252)
fix #14250

Signed-off-by: a1012112796 <1012112796@qq.com>
2021-01-09 17:34:08 +00:00
Lunny Xiao 3c96a37162
Some code improvements (#14266) 2021-01-06 19:23:57 +00:00
Lunny Xiao 9f73cae635
Fix wrong type on hooktask to convert typ from char(16) to varchar(16) (#14148)
* Fix wrong type on hooktask to convert typ from char(16) to varchar(16)

* Fix bugs

* Improve code

* Use different trim function for MSSQL

* Fix bug

* Removed wrong changed line

* Removed wrong changed line

* Fix nullable

* Fix lint

* Ignore sqlite on migration

* Fix mssql modify column failure

* Move modifyColumn to migrations.go so that other migrate function could use it
2021-01-06 16:11:23 +01:00
Jimmy Praet e6acce649b
Send notifications for mentions in pulls, issues, (code-)comments (#14218)
Fixes #14187: mention handling extracted from email notification code
Fixes #14013: add notification for mentions in pull request code comments
Fixes #13450: Not receiving any emails with setting "Only Email on Mention"
2021-01-02 18:04:02 +01:00
Lunny Xiao 8c086bae6e
Fix wrong publisher id (#14212)
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: zeripath <art27@cantab.net>
2021-01-01 16:03:31 -05:00
Chester Liu 632800eda7
Check for 'main' as potential default branch name (#14193) 2020-12-30 15:46:26 +00:00
zeripath 236e70f135
Fix escaping issue in diff (#14153)
Ensure that linecontent is escaped before passing to template.HTML

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-12-26 21:58:21 +00:00
6543 a19447aed1
migrate from com.* to alternatives (#14103)
* remove github.com/unknwon/com from models

* dont use "com.ToStr()"

* replace "com.ToStr" with "fmt.Sprint" where its easy to do

* more refactor

* fix test

* just "proxy" Copy func for now

* as per @lunny
2020-12-25 11:59:32 +02:00
Cirno the Strongest 2c9dd71140
Standardize Co-Authored-By / Reviewed-By strings (#14097)
* Standardize Co-Authored-By / Reviewed-By strings

* Use lowercase variant

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2020-12-21 21:19:33 -05:00
Mura Li 09304db9a5
Use the text of pull-request as the squash commit's message (#13071)
Originally, it was filled by the commit messages of the involved
commits. In this change, we use the headline comment of the pull
request as the commit message when it is a squash merge.

Thanks to @zeripath for suggesting the idea.

Fixes #12365

Co-authored-by: Mura Li <typeless@users.noreply.github.com>
2020-12-21 17:46:14 +01:00
Cirno the Strongest f3c4baa84b
Show dropdown with all statuses for commit (#13977)
* Show dropdown with all statuses for commit

* Use popups

* Remove unnecessary change

* Style popup

* Use divided list

* As per @silverwind

* Refactor GetLastCommitStatus

* Missing dropdown on repo home and commit page

* Fix tests

* Make status icon be a part of a link on PR list

* Fix missing translation call

* Indent fix

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-12-20 11:13:12 +08:00
Cirno the Strongest efa9a8a6e3
Show status check for merged PRs (#13975)
* Show status check for merged PRs

* Handle PRs with no commits

* Styling

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: 6543 <6543@obermui.de>
2020-12-18 13:37:55 +01:00
6543 e483220ea3
[Refactor] CombinedStatus and CommitStatus related functions & structs (#14026)
* RM unused struct

* rename (*CommitStatus) loadRepo() -> loadAttributes()

* move ToCommitStatus into its own file

* use CommitStatusState instead of StatusState

* move CombinedStatus convertion into convert package

* let models.GetLatestCommitStatus use repoID direct and accept ListOptions

* update swagger docs

* fix tests

* Fix swagger docs

* rm page

* fix swagger docs!!!

* return json null

* always return json

* rename api.Status to api.CommitStatus

* fix swagger docs

* sec swagger fix
2020-12-18 11:33:32 +08:00
Cirno the Strongest 53308de0bf
Send webhook when tag is removed via Web UI (#14015)
* Send webhook when tag is removed via Web UI

* Stray code
2020-12-16 10:00:46 -06:00
Cirno the Strongest 66379ba7ae
Send push event when tag is created or deleted (#13999)
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2020-12-16 14:41:21 +02:00
Jimmy Praet e7a77d32cc
Fix correct diff view for PR review comments in PR view page (#14002)
Fixes #13683.

The diff snippet that provides context for a code review comment on the pull request timeline page used to be calculated based on the headCommitID. But in 1.13, with PR #13448, this changed to the commitID from the blame for the commented line, which seems to cause these incorrect review comment diff snippets.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2020-12-16 11:54:58 +02:00
Lunny Xiao 9f100a45c6
Possible fix the webhook API creation (#13960)
* Possible fix the webhook API creation

* Fix api create webhook bug
2020-12-12 16:33:19 +01:00
zeripath f9c0737a86
Enforce setting HEAD in wiki to master (#13950)
The default branch in wikis must be master - therefore forcibly set the HEAD
to master.

Fix #13846

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2020-12-12 12:20:14 +01:00
zeripath f2f99a7d0c
Ensure that pusher is not nil preventing panic in push (#13945)
PR #13381 refactored notification actions to call NotifyCreateRef and
NotifyDeleteRef in a slightly different manner. This leads to the pusher
not being set before the call.

This PR ensures that the pusher is updated before the call.

Fix #13940

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-12-11 18:27:00 +01:00
Lunny Xiao ea914d0f7a
Fix bug for webhook and feishu caused by API changed (#13937)
* Fix bug for webhook

* Fix bug for feishu
2020-12-11 17:04:04 +01:00
Lunny Xiao 6edfa6bc88
Fix broken migration on webhook (#13911)
* Fix broken migration on webhook

* Fix lint

Co-authored-by: John Olheiser <john.olheiser@gmail.com>
2020-12-09 17:20:13 +00:00
Lunny Xiao 42354dfe45
Move webhook type from int to string (#13664)
* Move webhook type from int to string

* rename webhook_services

* finish refactor

* Fix merge

* Ignore unnecessary ci

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
2020-12-08 11:41:14 +01:00
Lunny Xiao ab22ab4a37
Refactor push update (#13381)
* Refactor Push update

* Remove the push_test since the function has been removed.

* Use default branch setting instead master
2020-12-08 03:23:18 +01:00
a1012112796 682f0b0460
Fix Pull Merge when tag with same name as base branch exist (#13882)
fix dst refspec error in 'Push back to upstream' when base branch have
same name with a tag.

fix #13851
Signed-off-by: a1012112796 <1012112796@qq.com>

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
2020-12-08 02:46:50 +01:00
manuelluis 8561f26695
Fix branch/tag notifications in mirror sync (#13855)
* Fix branch/tag notifications in mirror sync

* Fix gofmt

Co-authored-by: Gitea <gitea@fake.local>
Co-authored-by: Lauris BH <lauris@nix.lv>
2020-12-05 23:13:11 +08:00
John Olheiser f0cc29761d
Make sure email recipients can see issue (#13820)
* Initial pass

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Remove over-op

Signed-off-by: jolheiser <john.olheiser@gmail.com>
2020-12-03 14:25:49 -06:00
6543 2b4a08e962
Migrations: Use Process Manager to create own Context (#13792) 2020-12-02 18:36:06 +00:00
a1012112796 9c26dc1f3a
Add block on official review requests branch protection (#13705)
Signed-off-by: a1012112796 <1012112796@qq.com>

Co-authored-by: Lauris BH <lauris@nix.lv>
2020-11-28 21:30:46 +02:00
zeripath 742e21aeba
Handle and propagate errors when checking if paths are Dirs, Files or Exist (#13186)
* Ensure errors from IsDir propagate

* Handle errors when checking IsFile

* Handle and propagate errors from IsExist

* Update modules/templates/static.go

* Update modules/templates/static.go

* Return after ctx.ServerError

* Apply suggestions from code review

* Fix tests

The previous merge managed to break repo_form.go

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
2020-11-27 21:42:08 -05:00
John Olheiser 979fbe086c
Increment skip to avoid infini-loop (#13703)
Signed-off-by: jolheiser <john.olheiser@gmail.com>

Co-authored-by: Lauris BH <lauris@nix.lv>
2020-11-27 22:00:52 +02:00
Jimmy Praet 03fa2eccbc
Use chronological commit order in default squash message (#13661) (#13696) 2020-11-25 15:08:17 -05:00
zeripath 45ca2e4605
Handle incomplete diff files properly (#13662)
* Handle incomplete diff files properly

The code for parsing diff hunks has a bug whereby a very long line
in a very long diff would not be completely read leading to an unexpected
character.

This PR ensures that the line is completely cleared

Fix #13602

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Also allow git max line length <4096

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add test case

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-11-21 17:41:24 -05:00
zeripath b091c994b5
Comments on review should have the same sha (#13448)
* When replying to an outdated comment it should not appear on the files page

This happened because the comment took the latest commitID as its base instead of the
reviewID that it was replying to.

There was also no way of creating an already outdated comment - and a
reply to a review on an outdated line should be outdated.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix test

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Fix broken migration

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix mssql

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Create temporary table because ... well MSSQL ...

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Create temporary table because ... well MSSQL ...

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Create temporary table because ... well MSSQL ...

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix mssql

Signed-off-by: Andrew Thornton <art27@cantab.net>

* move session within the batch

Signed-off-by: Andrew Thornton <art27@cantab.net>

* regen the sqlcmd each time round the loop

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per @lunny

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2020-11-09 01:15:09 -05:00
Kyle Evans e461f0854f
[RFC] Make archival asynchronous (#11296)
* Make archival asynchronous

The prime benefit being sought here is for large archives to not
clog up the rendering process and cause unsightly proxy timeouts.
As a secondary benefit, archive-in-progress is moved out of the
way into a /tmp file so that new archival requests for the same
commit will not get fulfilled based on an archive that isn't yet
finished.

This asynchronous system is fairly primitive; request comes in, we'll
spawn off a new goroutine to handle it, then we'll mark it as done.
Status requests will see if the file exists in the final location,
and report the archival as done when it exists.

Fixes #11265

* Archive links: drop initial delay to three-quarters of a second

Some, or perhaps even most, archives will not take all that long to archive.
The archive process starts as soon as the download button is initially
clicked, so in theory they could be done quite quickly.  Drop the initial
delay down to three-quarters of a second to make it more responsive in the
common case of the archive being quickly created.

* archiver: restructure a little bit to facilitate testing

This introduces two sync.Cond pointers to the archiver package. If they're
non-nil when we go to process a request, we'll wait until signalled (at all)
to proceed. The tests will then create the sync.Cond so that it can signal
at-will and sanity-check the state of the queue at different phases.

The author believes that nil-checking these two sync.Cond pointers on every
archive processing will introduce minimal overhead with no impact on
maintainability.

* gofmt nit: no space around binary + operator

* services: archiver: appease golangci-lint, lock queueMutex

Locking/unlocking the queueMutex is allowed, but not required, for
Cond.Signal() and Cond.Broadcast().  The magic at play here is just a little
too much for golangci-lint, as we take the address of queueMutex and this is
mostly used in archiver.go; the variable still gets flagged as unused.

* archiver: tests: fix several timing nits

Once we've signaled a cond var, it may take some small amount of time for
the goroutines released to hit the spot we're wanting them to be at. Give
them an appropriate amount of time.

* archiver: tests: no underscore in var name, ungh

* archiver: tests: Test* is run in a separate context than TestMain

We must setup the mutex/cond variables at the beginning of any test that's
going to use it, or else these will be nil when the test is actually ran.

* archiver: tests: hopefully final tweak

Things got shuffled around such that we carefully build up and release
requests from the queue, so we can validate the state of the queue at each
step. Fix some assertions that no longer hold true as fallout.

* repo: Download: restore some semblance of previous behavior

When archival was made async, the GET endpoint was only useful if a previous
POST had initiated the download. This commit restores the previous behavior,
to an extent; we'll now submit the archive request there and return a
"202 Accepted" to indicate that it's processing if we didn't manage to
complete the request within ~2 seconds of submission.

This lets a client directly GET the archive, and gives them some indication
that they may attempt to GET it again at a later time.

* archiver: tests: simplify a bit further

We don't need to risk failure and use time.ParseDuration to get 2 *
time.Second.

else if isn't really necessary if the conditions are simple enough and lead
to the same result.

* archiver: tests: resolve potential source of flakiness

Increase all timeouts to 10 seconds; these aren't hard-coded sleeps, so
there's no guarantee we'll actually take that long. If we need longer to
not have a false-positive, then so be it.

While here, various assert.{Not,}Equal arguments are flipped around so that
the wording in error output reflects reality, where the expected argument is
second and actual third.

* archiver: setup infrastructure for notifying consumers of completion

This API will *not* allow consumers to subscribe to specific requests being
completed, just *any* request being completed. The caller is responsible for
determining if their request is satisfied and waiting again if needed.

* repo: archive: make GET endpoint synchronous again

If the request isn't complete, this endpoint will now submit the request and
wait for completion using the new API. This may still be susceptible to
timeouts for larger repos, but other endpoints now exist that the web
interface will use to negotiate its way through larger archive processes.

* archiver: tests: amend test to include WaitForCompletion()

This is a trivial one, so go ahead and include it.

* archiver: tests: fix test by calling NewContext()

The mutex is otherwise uninitialized, so we need to ensure that we're
actually initializing it if we plan to test it.

* archiver: tests: integrate new WaitForCompletion a little better

We can use this to wait for archives to come in, rather than spinning and
hoping with a timeout.

* archiver: tests: combine numQueued declaration with next-instruction assignment

* routers: repo: reap unused archiving flag from DownloadStatus()

This had some planned usage before, indicating whether this request
initiated the archival process or not. After several rounds of refactoring,
this use was deemed not necessary for much of anything and got boiled down
to !complete in all cases.

* services: archiver: restructure to use a channel

We now offer two forms of waiting for a request:
- WaitForCompletion: wait for completion with no timeout
- TimedWaitForCompletion: wait for completion with timeout

In both cases, we wait for the given request's cchan to close; in the latter
case, we do so with the caller-provided timeout. This completely removes the
need for busy-wait loops in Download/InitiateDownload, as it's fairly clean
to wait on a channel with timeout.

* services: archiver: use defer to unlock now that we can

This previously carried the lock into the goroutine, but an intermediate
step just added the request to archiveInProgress outside of the new
goroutine and removed the need for the goroutine to start out with it.

* Revert "archiver: tests: combine numQueued declaration with next-instruction assignment"

This reverts commit bcc52140238e16680f2e05e448e9be51372afdf5.

Revert "archiver: tests: integrate new WaitForCompletion a little better"

This reverts commit 9fc8bedb5667d24d3a3c7843dc28a229efffb1e6.

Revert "archiver: tests: fix test by calling NewContext()"

This reverts commit 709c35685eaaf261ebbb7d3420e3376a4ee8e7f2.

Revert "archiver: tests: amend test to include WaitForCompletion()"

This reverts commit 75261f56bc05d1fa8ff7e81dcbc0ccd93fdc9d50.

* archiver: tests: first attempt at WaitForCompletion() tests

* archiver: tests: slight improvement, less busy-loop

Just wait for the requests to complete in order, instead of busy-waiting
with a timeout.  This is slightly less fragile.

While here, reverse the arguments of a nearby assert.Equal() so that
expected/actual are correct in any test output.

* archiver: address lint nits

* services: archiver: only close the channel once

* services: archiver: use a struct{} for the wait channel

This makes it obvious that the channel is only being used as a signal,
rather than anything useful being piped through it.

* archiver: tests: fix expectations

Move the close of the channel into doArchive() itself; notably, before these
goroutines move on to waiting on the Release cond.

The tests are adjusted to reflect that we can't WaitForCompletion() after
they've already completed, as WaitForCompletion() doesn't indicate that
they've been released from the queue yet.

* archiver: tests: set cchan to nil for comparison

* archiver: move ctx.Error's back into the route handlers

We shouldn't be setting this in a service, we should just be validating the
request that we were handed.

* services: archiver: use regex to match a hash

This makes sure we don't try and use refName as a hash when it's clearly not
one, e.g. heads/pull/foo.

* routers: repo: remove the weird /archive/status endpoint

We don't need to do this anymore, we can just continue POSTing to the
archive/* endpoint until we're told the download's complete. This avoids a
potential naming conflict, where a ref could start with "status/"

* archiver: tests: bump reasonable timeout to 15s

* archiver: tests: actually release timedReq

* archiver: tests: run through inFlight instead of manually checking

While we're here, add a test for manually re-processing an archive that's
already been complete. Re-open the channel and mark it incomplete, so that
doArchive can just mark it complete again.

* initArchiveLinks: prevent default behavior from clicking

* archiver: alias gitea's context, golang context import pending

* archiver: simplify logic, just reconstruct slices

While the previous logic was perhaps slightly more efficient, the
new variant's readability is much improved.

* archiver: don't block shutdown on waiting for archive

The technique established launches a goroutine to do the wait,
which will close a wait channel upon termination. For the timeout
case, we also send back a value indicating whether the timeout was
hit or not.

The timeouts are expected to be relatively small, but still a multi-
second delay to shutdown due to this could be unfortunate.

* archiver: simplify shutdown logic

We can just grab the shutdown channel from the graceful manager instead of
constructing a channel to halt the caller and/or pass a result back.

* Style issues

* Fix mis-merge

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
2020-11-07 22:27:28 +02:00
Lunny Xiao 35cc82abbf
Revert "Replies to outdated code comments should also be outdated (#13217)" (#13439)
This reverts commit 3cab3bee57.
2020-11-06 09:04:21 +02:00
mrsdizzie d7e0983780
Alternative fix for HTML diff entity split (#13425)
* Alternative fix for HTML diff entity split

This commit both reverts PR #13357 and uses the exiting implementation alredy used for spans to fix the same issue. That PR duplicates most of logic that is already present elsewhere and still was failing for some cases. This should be simpler as it uses the existing logic that already works for <span>s being split apart.

Added both test cases as well.

* Update gitdiff_test.go

* fmt

* entity can have uppercase letter, also add detailed comment per @zeripath
2020-11-04 19:35:05 -05:00