GitLab实战十二——Gitlab 9.3.11 升级至 10.0.6

Gitlab10新特性

• Web IDE开源，Markdown

• Deploy Tokens部署令牌，增量部署

• Incremental rollout deployments（增量部署）

当软件需要发生重大更改时，即使有严格的预发布测试，对生产代码库的更改也可能导致无法预料的影响。因此，你可能需要将最新版本部署给一小部分的用户，以获取反馈并确保不存在任何问题。

Incremental rollout deployments 功能可以帮助开发者只对一部分用户部署新代码，并可以分步重复此过程，从而增加使用新版本的用户的百分比。当问题发生，可以回滚以前的老版本而不影响整个用户群

• Push Mirroring 开源

自发布以来一直是付费使用的 Push Mirroring 在此版本正式开源。存储库镜像允许你将 Git 存储库从一个位置复制到另一个位置，这使得使用多个 GitLab 实例更容易。通过 push 镜像，还可以更轻松地将项目从其他位置移动到 GitLab ，而不会中断旧版本库的更新

• 基于变量的CI/CD流控制

• SAST 已支持 Go 和 C / C ++ 语言

  静态应用安全测试（SAST）是 GitLab 提供的开箱即用的安全工具之一，用以分析源代码以了解已知漏洞，并直接在合并请求上输出结果以便于查看。目前再次扩大范围，正式支持 Go 和 C / C ++

• 安全报告交互式反馈

安全报告会显示哪个漏洞可能会影响你的软件，但你需要采取相应的措施解决这些漏洞并确保产品安全。GitLab 10.8 之后，你可以直接从安全报告中创建一个 issue 并解决漏洞。如果这是一个误报，你也可以关闭该条目

执行升级步骤：

/etc/init.d/gitlab stop

1. Get latest code

sudo -u git -H git fetch --all

sudo -u git -H git checkout -- db/schema.rb  # local changes will be restored automatically

sudo -u git -H git checkout -- locale

sudo -u git -H git checkout -f 10-0-stable

2.  Update gitlab-shell

cd /home/git/gitlab-shell/

sudo -u git -H git fetch --all

sudo -u git -H git checkout v$(</home/git/git/GITLAB_SHELL_VERSION)

sudo -u git -H bin/compile

Previous HEAD position was 0a64624... Bump version to v5.1.1

HEAD is now at 2b575a8... Bump version to 5.9.0

3.  Update gitlab-workhorse

cd /home/git/gitlab-workhorse/

sudo -u git -H git fetch --all

sudo -u git -H git checkout v$(</home/git/git/GITLAB_WORKHORSE_VERSION)

sudo -u git -H make

Previous HEAD position was e36227d... Merge branch 'release-2.1.1' into 'master'

HEAD is now at e6cfe0c... Merge branch 'version-2.4.0' into 'master'

4.  Update Gitaly

cd /home/git/gitaly/

sudo -u git -H git fetch --all

sudo -u git -H git checkout v$(</home/git/git/GITALY_SERVER_VERSION)

sudo -u git -H make

Previous HEAD position was e8f0485... Version 0.11.2

HEAD is now at 137cfb9... Version 0.38.0

MySQL permissions授权语句之前已经执行过了，这里就过了没执行可以看官方文档

mysql -u root -p -e "GRANT TRIGGER ON `gitlabhq_production`.* TO 'git'@'localhost';"

mysql -u root -p -e "SET GLOBAL log_bin_trust_function_creators = 1;"

5. Update configuration files

先把gitlab主配置文件gitlab.yml和gitaly的配置文件config.toml都备份

sudo -u git cp /home/git/git/config/gitlab.yml.example /home/git/git/config/gitlab.yml

编辑gitlab.yml改成原来相应的配置

sudo -u git vim /home/git/git/config/gitlab.yml

sudo -u git cp /home/git/gitaly/config.toml.example /home/git/gitaly/config.toml

sudo -u git vim /home/git/gitaly/config.toml

socket_path = "/home/git/git/tmp/sockets/private/gitaly.socket"

[[storage]]

name = "default"

path = "/home/git/repositories"

[gitaly-ruby]

dir = "/home/git/git/ruby"

[gitlab-shell]

dir = "/home/git/gitlab-shell"

6. Install libs, migrations, etc.

cd /home/git/git

cp /etc/init.d/gitlab{,.bak-9.3.11}

sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab

vim /etc/init.d/gitlab

sed -i "s|^source .*|source 'https://gems.ruby-china.com/'|g" Gemfile

sed -i "s|remote.*|remote: https://gems.ruby-china.com/|g" Gemfile.lock

sudo -u git -H bundle update

# MySQL installations (note: the line below states '--without postgres')
sudo -u git -H bundle install --without postgres development test --deployment

# Optional: clean up old gems
sudo -u git -H bundle clean

# Run database migrations
sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production

# Compile GetText PO files

sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production

# Update node dependencies and recompile assets
sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production

# Clean up cache
sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production

Mysql2::Error: SELECT command denied to user 'gitlab_w'@'10.162.21.48' for table 'user': SHOW FULL FIELDS FROM mysql.user

登陆数据库执行：

GRANT select ON `mysql`.user TO 'gitlab_w'@'10.%';

执行后继续报错：
== 20170629171610 RenameApplicationSettingsSigninEnabledToPasswordAuthenticationEnabled: migrating
-- transaction_open?()
  -> 0.0000s
rake aborted!
StandardError: An error has occurred, all later migrations canceled:

Your database user is not allowed to create, drop, or execute triggers on the
table application_settings.

If you are using PostgreSQL you can solve this by logging in to the GitLab
database (gitlabhq_production) using a super user and running:

ALTER gitlab_w WITH SUPERUSER

For MySQL you instead need to run:

GRANT ALL PRIVILEGES ON *.* TO gitlab_w@'%'

执行：GRANT ALL PRIVILEGES ON *.* TO gitlab_w@'%'

-- execute("CREATE TRIGGER trigger_0b1f9aeacd95_insert\nBEFORE INSERT\nON `application_settings`\nFOR EACH ROW\nSET NEW.`password_authentication_enabled` = NEW.`signin_enabled`\n")
  -> 0.0191s
-- execute("CREATE TRIGGER trigger_0b1f9aeacd95_update\nBEFORE UPDATE\nON `application_settings`\nFOR EACH ROW\nSET NEW.`password_authentication_enabled` = NEW.`signin_enabled`\n")


Mysql2::Error: Illegal mix of collations (utf8_general_ci,IMPLICIT) and (utf8_unicode_ci,IMPLICIT) for operation '=':         INSERT INTO user_synced_attributes_metadata (user_id, provider, email_synced)
       SELECT id, email_provider, external_email
       FROM users
       WHERE external_email = TRUE
       AND NOT EXISTS (
         SELECT true
         FROM user_synced_attributes_metadata
         WHERE user_id = users.id
         AND (provider = users.email_provider OR (provider IS NULL AND users.email_provider IS NULL))
       )
       AND id BETWEEN 1 AND 1014


解决：
alter table user_synced_attributes_metadata modify `provider` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL;

数据库表结构变更

Mysql2::Error: Illegal mix of collations (utf8_general_ci,IMPLICIT) and (utf8_unicode_ci,IMPLICIT) for operation '=':         INSERT INTO user_synced_attributes_metadata (user_id, provider, email_synced)
       SELECT id, email_provider, external_email
       FROM users
       WHERE external_email = TRUE
       AND NOT EXISTS (
         SELECT true
         FROM user_synced_attributes_metadata
         WHERE user_id = users.id
         AND (provider = users.email_provider OR (provider IS NULL AND users.email_provider IS NULL))
       )
       AND id BETWEEN 1 AND 1014

解决：
alter table user_synced_attributes_metadata modify `provider` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL;


Mysql2::Error: Specified key was too long; max key length is 767 bytes: CREATE UNIQUE INDEX `index_ci_group_variables_on_group_id_and_key`  ON `ci_group_variables` (`group_id`, `key`)

alter table ci_group_variables modify column `key` varchar(190);

vim /home/git/git/db/migrate/20170525130346_create_group_variables_table.rb

Mysql2::Error: Specified key was too long; max key length is 767 bytes: CREATE UNIQUE INDEX `index_ci_pipeline_schedule_variables_on_schedule_id_and_key`  ON `ci_pipeline_schedule_variables` (`pipeline_schedule_id`, `key`)

alter table ci_pipeline_schedule_variables modify column `key` varchar(190);

vim /home/git/git/db/migrate/20170620064728_create_ci_pipeline_schedule_variables.rb


Mysql2::Error: Specified key was too long; max key length is 767 bytes: CREATE UNIQUE INDEX `index_ci_pipeline_variables_on_pipeline_id_and_key`  ON `ci_pipeline_variables` (`pipeline_id`, `key`)

alter table ci_pipeline_variables modify column `key` varchar(190);

vim /home/git/git/db/migrate/20170720130522_create_ci_pipeline_variables.rb


== 20170830130119 StealRemainingEventMigrationJobs: migrating ================= 这一步执行n久，1小时以上
把events表数据作优化
select count(1) from events;200多万•行
delete from events where created_at<'2017-04-01' limit 100000;

select count(1) from push_event_payloads;

Mysql2::Error: Incorrect string value: '\xF0\x9F\x9A\x80 (...' for column 'commit_title' at row 1: INSERT INTO `push_event_payloads` (`event_id`, `commit_count`, `ref_type`, `action`, `commit_from`, `commit_to`, `ref`, `commit_title`) VALUES (2039457, 7, 0, 2, x'51848f3c5230fee8b777f94e7baf5d366ffda1ca', x'f51ac63717dea015bd9445da56710ddd5004c3e2', 'master', 'Update @angular-devkit/build-angular to the latest version ð••• (#3623)')

show variables like 'chara%';

show create table push_event_payloads \G;

ALTER TABLE push_event_payloads CONVERT TO CHARACTER SET utf8mb4;

ALTER TABLE push_event_payloads ROW_FORMAT=DYNAMIC;

# ALTER TABLE push_event_payloads MODIFY commit_title VARCHAR(70) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci DEFAULT NULL

INSERT INTO `push_event_payloads` (`event_id`, `commit_count`, `ref_type`, `action`, `commit_from`, `commit_to`, `ref`, `commit_title`) VALUES (1997505, 1, 0, 2, x'cf6f04d9e24feff4802f360cba5e2ebc1d4b043a', x'b7bb5a2780e590e0f265f7c5264a27887061fee9', 'master', 'Update git-describe to the latest version ð••• (#3560)');

INSERT INTO `push_event_payloads` (`event_id`, `commit_count`, `ref_type`, `action`, `commit_to`, `ref`, `commit_title`) VALUES (1981100, 18660, 0, 0, x'323d5f8e9418dbf138b19bd53464cfa5b9c26d80', 'er-ðer-ð®-off', 'Remove copy-on-write when caching bitmaps');

https://gitlab.com/gitlab-org/gitlab-ce/issues/38823

前端资源编译

sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production

执行报错：

warning "minimatch#brace-expansion@^1.0.0" could be deduped from "1.1.8" to "[email protected]"
error "webpack#ajv" not installed
error "webpack#ajv-keywords" not installed
error Found 2 errors.

yarn add webpack

chown -R git.git /home/git/git/node_modules/

error "babel-loader#webpack@2 || 3" doesn't satisfy found match of "[email protected]"

error "compression-webpack-plugin#webpack@^2.0.0 || ^3.0.0" doesn't satisfy found match of "[email protected]"

yarn install && chown -R git.git /home/git/git/node_modules/

yarn upgrade

warning "karma-webpack > [email protected]" has incorrect peer dependency "webpack@^1.0.0 || ^2.0.0 || ^3.0.0".
warning " > [email protected]" has incorrect peer dependency "webpack@^2.2.0 || ^3.0.0".

yarn add uglifyjs-webpack-plugin karma-webpack [email protected]

Error: webpack.optimize.CommonsChunkPlugin has been removed, please use config.optimization.splitChunks instead.
   at Object.get [as CommonsChunkPlugin] (/home/git/git/node_modules/webpack/lib/webpack.js:185:10)

因为webpack4.0开始摒弃了CommonsChunkPlugin，要把webpack.config.js里的new webpack.optimize.CommonsChunkPlugin修改为splitChunks。
因为我的是vue-cli项目，并没有找到webpack.config.js这个文件。。

然后我在webpack.prod.conf.js里发现了new webpack.optimize.CommonsChunkPlugin并修改为splitChunks后。build的时候会报 splitChunks is not defined。

编辑/home/git/git/config/webpack.config.js文件

webpack.optimize.CommonsChunkPlugin修改为webpack.optimize.SplitChunksPlugin（修改三处）

Error: webpack.optimize.UglifyJsPlugin has been removed, please use config.optimization.minimize instead.

webpack.optimize.UglifyJsPlugin 修改为

vim /home/git/git/config/webpack.config.js

error An unexpected error occurred: "expected hoisted manifest".

yarn add react-native-razorpay

都配置完了后启动gitlab，看日志有没有报错

error [email protected]: The engine "node" is incompatible with this module. Expected version ">=8.3". Got "6.14.3"
error Found incompatible module

yarn add npm

sudo npm cache clean -f

yarn add react-native

sudo npm install -g n

sudo n stable

[email protected]

[email protected]

error "npm#validate-npm-package-license#spdx-correct" is wrong version: expected "3.1.0", got "3.0.0"

npm i validate-npm-package-license

error "npm#rimraf" is wrong version: expected "2.6.3", got "2.6.2"

error Lockfile does not contain pattern: "rimraf@^2.6.3"

执行yarn

error "karma-webpack#webpack@^2.0.0 || ^3.0.0" doesn't satisfy found match of "[email protected]"

sudo -u git -H bundle exec rake assets:precompile RAILS_ENV=production

"nodejs": "8.11.3",
   "react-native": "0.57.1",

/usr/local/bin/node

yarn add [email protected]

yarn add ajv ajv-keywords

7.  Check application status

/etc/init.d/gitlab start

cd /home/git/git

sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production

sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production

检查都没错，启动也正常，通过域名访问也没有问题就可以测试相关功能了。

# 可以使用下面命令指定root密码,默认无密码，也可初次访问时修改。
sudo -u git -H bundle exec rake gitlab:setup RAILS_ENV=production GITLAB_ROOT_PASSWORD=yourpassword  GITLAB_ROOT_EMAIL=youremail

https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/update/8.17-to-9.0.md

Gitlab 10.0.6 前端模块组件

{
 "private": true,
 "scripts": {
   "dev-server": "nodemon --watch config/webpack.config.js -- ./node_modules/.bin/webpack-dev-server --config config/webpack.config.js",
   "eslint": "eslint --max-warnings 0 --ext .js,.vue .",
   "eslint-fix": "eslint --max-warnings 0 --ext .js,.vue --fix .",
   "eslint-report": "eslint --max-warnings 0 --ext .js,.vue --format html --output-file ./eslint-report.html .",
   "karma": "karma start config/karma.config.js --single-run",
   "karma-coverage": "BABEL_ENV=coverage karma start config/karma.config.js --single-run",
   "karma-start": "karma start config/karma.config.js",
   "webpack": "webpack --config config/webpack.config.js",
   "webpack-prod": "NODE_ENV=production webpack --config config/webpack.config.js"
 },
 "dependencies": {
   "ajv": "^6.6.2",
   "ajv-keywords": "^3.2.0",
   "axios": "^0.16.2",
   "babel-core": "^6.22.1",
   "babel-eslint": "^7.2.1",
   "babel-loader": "^7.1.1",
   "babel-plugin-transform-define": "^1.2.0",
   "babel-preset-latest": "^6.24.0",
   "babel-preset-stage-2": "^6.22.0",
   "bootstrap-sass": "^3.3.6",
   "brace-expansion": "^1.1.8",
   "compression-webpack-plugin": "^1.0.0",
   "copy-webpack-plugin": "^4.0.1",
   "core-js": "^2.4.1",
   "cropper": "^2.3.0",
   "css-loader": "^0.28.0",
   "d3": "^3.5.11",
   "deckar01-task_list": "^2.0.0",
   "document-register-element": "^1.3.0",
   "dropzone": "^4.2.0",
   "emoji-unicode-version": "^0.2.1",
   "eslint-plugin-html": "^2.0.1",
   "exports-loader": "^0.6.4",
   "file-loader": "^0.11.1",
   "imports-loader": "^0.7.1",
   "jed": "^1.1.1",
   "jquery": "^2.2.1",
   "jquery-ujs": "^1.2.1",
   "js-cookie": "^2.1.3",
   "jszip": "^3.1.3",
   "jszip-utils": "^0.0.2",
   "marked": "^0.3.6",
   "monaco-editor": "0.8.3",
   "mousetrap": "^1.4.6",
   "name-all-modules-plugin": "^1.0.1",
   "pikaday": "^1.5.1",
   "prismjs": "^1.6.0",
   "raphael": "^2.2.7",
   "raven-js": "^3.14.0",
   "raw-loader": "^0.5.1",
   "react-dev-utils": "^0.5.2",
   "select2": "3.5.2-browserify",
   "sql.js": "^0.4.0",
   "three": "^0.84.0",
   "three-orbit-controls": "^82.1.0",
   "three-stl-loader": "^1.0.4",
   "timeago.js": "^2.0.5",
   "underscore": "^1.8.3",
   "url-loader": "^0.5.8",
   "visibilityjs": "^1.2.4",
   "vue": "^2.2.6",
   "vue-loader": "^11.3.4",
   "vue-resource": "^1.3.4",
   "vue-template-compiler": "^2.2.6",
   "vuex": "^2.3.1",
   "webpack": "3.5.5",
   "webpack-bundle-analyzer": "^2.8.2",
   "webpack-stats-plugin": "^0.1.5"
 },
 "devDependencies": {
   "babel-plugin-istanbul": "^4.0.0",
   "eslint": "^3.10.1",
   "eslint-config-airbnb-base": "^10.0.1",
   "eslint-import-resolver-webpack": "^0.8.3",
   "eslint-plugin-filenames": "^1.1.0",
   "eslint-plugin-import": "^2.2.0",
   "eslint-plugin-jasmine": "^2.1.0",
   "eslint-plugin-promise": "^3.5.0",
   "istanbul": "^0.4.5",
   "jasmine-core": "^2.6.3",
   "jasmine-jquery": "^2.1.1",
   "karma": "^1.7.0",
   "karma-chrome-launcher": "^2.1.1",
   "karma-coverage-istanbul-reporter": "^0.2.0",
   "karma-jasmine": "^1.1.0",
   "karma-mocha-reporter": "^2.2.2",
   "karma-sourcemap-loader": "^0.3.7",
   "karma-webpack": "^2.0.4",
   "nodemon": "^1.11.0",
   "webpack-dev-server": "^2.6.1"
 }
}

升级后验证