[Starlingx-discuss] error when installing starlingx3.0 all-in-one in bare metal

Sun Apr 19 08:16:33 UTC 2020

The machine belongs to my school. How rich a school it is! :-)

I have tried that method. 

Without any change, the result of "cat /sys/block/sda/device/../*/scsi_disk/*/max_write_same_blocks" is already 0.

BTW, the puppy log has some errors, but I think the most important one is listed below:
===============================================
2020-04-18T03:07:22.686 Error: 2020-04-18 03:07:22 +0000 drbdadm -- --assume-peer-has-space resize drbd-dockerdistribution returned 10 instead of one of [0]
2020-04-18T03:07:22.691 /usr/share/ruby/vendor_ruby/puppet/util/errors.rb:106:in `fail'
2020-04-18T03:07:22.696 /usr/share/ruby/vendor_ruby/puppet/type/exec.rb:160:in `sync'
2020-04-18T03:07:22.702 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:236:in `sync'
2020-04-18T03:07:22.709 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:134:in `sync_if_needed'
2020-04-18T03:07:22.715 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:88:in `block in perform_changes'
2020-04-18T03:07:22.721 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:87:in `each'
2020-04-18T03:07:22.727 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:87:in `perform_changes'
2020-04-18T03:07:22.732 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:21:in `evaluate'
2020-04-18T03:07:22.739 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:230:in `apply'
2020-04-18T03:07:22.744 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:246:in `eval_resource'
2020-04-18T03:07:22.750 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:163:in `call'
2020-04-18T03:07:22.756 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:163:in `block (2 levels) in evaluate'
2020-04-18T03:07:22.761 /usr/share/ruby/vendor_ruby/puppet/util.rb:386:in `block in thinmark'
2020-04-18T03:07:22.768 /usr/share/ruby/benchmark.rb:296:in `realtime'
2020-04-18T03:07:22.774 /usr/share/ruby/vendor_ruby/puppet/util.rb:385:in `thinmark'
2020-04-18T03:07:22.781 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:163:in `block in evaluate'
2020-04-18T03:07:22.787 /usr/share/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:in `traverse'
2020-04-18T03:07:22.792 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:154:in `evaluate'
2020-04-18T03:07:22.799 /usr/share/ruby/vendor_ruby/puppet/resource/catalog.rb:222:in `block in apply'
2020-04-18T03:07:22.806 /usr/share/ruby/vendor_ruby/puppet/util/log.rb:155:in `with_destination'
2020-04-18T03:07:22.812 /usr/share/ruby/vendor_ruby/puppet/transaction/report.rb:142:in `as_logging_destination'
2020-04-18T03:07:22.818 /usr/share/ruby/vendor_ruby/puppet/resource/catalog.rb:221:in `apply'
2020-04-18T03:07:22.824 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:171:in `block in apply_catalog'
2020-04-18T03:07:22.830 /usr/share/ruby/vendor_ruby/puppet/util.rb:223:in `block in benchmark'
2020-04-18T03:07:22.837 /usr/share/ruby/benchmark.rb:296:in `realtime'
2020-04-18T03:07:22.843 /usr/share/ruby/vendor_ruby/puppet/util.rb:222:in `benchmark'
2020-04-18T03:07:22.850 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:170:in `apply_catalog'
2020-04-18T03:07:22.855 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:343:in `run_internal'
2020-04-18T03:07:22.861 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:221:in `block in run'
2020-04-18T03:07:22.867 /usr/share/ruby/vendor_ruby/puppet/context.rb:65:in `override'
2020-04-18T03:07:22.874 /usr/share/ruby/vendor_ruby/puppet.rb:241:in `override'
2020-04-18T03:07:22.880 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:195:in `run'
2020-04-18T03:07:22.887 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:350:in `apply_catalog'
2020-04-18T03:07:22.893 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:274:in `block in main'
2020-04-18T03:07:22.900 /usr/share/ruby/vendor_ruby/puppet/context.rb:65:in `override'
2020-04-18T03:07:22.906 /usr/share/ruby/vendor_ruby/puppet.rb:241:in `override'
2020-04-18T03:07:22.912 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:225:in `main'
2020-04-18T03:07:22.918 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:170:in `run_command'
2020-04-18T03:07:22.923 /usr/share/ruby/vendor_ruby/puppet/application.rb:344:in `block in run'
2020-04-18T03:07:22.929 /usr/share/ruby/vendor_ruby/puppet/util.rb:540:in `exit_on_fail'
2020-04-18T03:07:22.935 /usr/share/ruby/vendor_ruby/puppet/application.rb:344:in `run'
2020-04-18T03:07:22.941 /usr/share/ruby/vendor_ruby/puppet/util/command_line.rb:132:in `run'
2020-04-18T03:07:22.946 /usr/share/ruby/vendor_ruby/puppet/util/command_line.rb:72:in `execute'
2020-04-18T03:07:22.952 /usr/bin/puppet:5:in `<main>'
2020-04-18T03:07:22.957 Error: 2020-04-18 03:07:22 +0000 /Stage[main]/Platform::Drbd::Dockerdistribution::Bootstrap/Platform::Drbd::Filesystem[drbd-dockerdistribution]/Exec[drbd resize drbd-dockerdistribution]/returns: change from notrun to 0 failed: drbdadm -- --assume-peer-has-space resize drbd-dockerdistribution returned 10 instead of one of [0]
======================================================
It seems that drbdadm doesn't work. And I am not sure what is Error 10 is. I wonder if you have some hints,

在 2020-04-19 16:01:31，"Sun, Austin" <austin.sun at intel.com> 写道：

Sda is so big. rich man :)

Get some search result from web [1]

Would you like check the result via the command

cat /sys/block/sda/device/../*/scsi_disk/*/max_write_same_blocks

it is 0 in my setup .   if it is 1 , you might change to 0 and try .

disable write same like this:

echo 0 | tee /sys/block/*/device/../*/scsi_disk/*/max_write_same_blocks

(for the relevant backend devices)

If you use LVM, you may need to vgchange -an ; vgchange -ay after that,

(at least for the relevant VGs), if they have already been activated.

[1] https://lists.linbit.com/pipermail/drbd-user/2017-February/023024.html

Thanks.

BR
Austin Sun.

From:周擎阳 <18245043314 at 163.com>
Sent: Sunday, April 19, 2020 3:38 PM
To: Sun, Austin <austin.sun at intel.com>
Cc: starlingx-discuss at lists.starlingx.io
Subject: Re:RE: Re:RE: [Starlingx-discuss] error when installing starlingx3.0 all-in-one in bare metal

4.9T is sda size.

the "lsblk" on my startlingx looks like this:

在 2020-04-19 15:14:33，"Sun, Austin" <austin.sun at intel.com> 写道：

is this 4.9T sda size or all disks size ?

how much sda is ?

Thanks.

BR
Austin Sun.

From:周擎阳 <18245043314 at 163.com>
Sent: Sunday, April 19, 2020 2:36 PM
To: Sun, Austin <austin.sun at intel.com>
Cc:starlingx-discuss at lists.starlingx.io
Subject: Re:RE: [Starlingx-discuss] error when installing starlingx3.0 all-in-one in bare metal

Dear Sir:

1.)  I am installing in bare metal environment, funny thing is that the process works when I install it in virtual environment in the same machine. The memory is 250G and the disk is 4.9T. 

2.)  the localhost.yml is just the one in the guide. Here is the detail:

===============================================================

system_mode: simplex

dns_servers:

  - 8.8.8.8

  - 8.8.4.4

external_oam_subnet: 172.16.0.0/16

external_oam_gateway_address: 172.16.0.1

external_oam_floating_address: 172.16.10.205

admin_username: admin

admin_password: ASdf1234!@#$

ansible_become_pass: ASdf1234!@#$

# Add these lines to configure Docker to use a proxy server

# docker_http_proxy: http://my.proxy.com:1080

# docker_https_proxy: https://my.proxy.com:1443

# docker_no_proxy:

#   - 1.2.3.4

===============================================================

3.) For the log file is too long, I simply attach it to this email. For convenience I simply put the error part of the log file below:

===============================================================

2020-04-18T03:10:32.010 Error: 2020-04-18 03:10:32 +0000 Failed to apply catalog: Execution of '/usr/bin/openstack --os-interface internal role list --quiet --format csv' returned 1: Unable to establish connection to http://127.0.0.1:5000/v3/roles?: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: /v3/roles (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f416632e190>: Failed to establish a new connection: [Errno 111] Connection refused',)) (tried 44, for a total of 170 seconds)

2020-04-18T03:10:32.017 /usr/share/openstack-puppet/modules/openstacklib/lib/puppet/provider/openstack.rb:126:in `rescue in block (2 levels) in request'

2020-04-18T03:10:32.021 /usr/share/openstack-puppet/modules/openstacklib/lib/puppet/provider/openstack.rb:88:in `block (2 levels) in request'

2020-04-18T03:10:32.026 /usr/share/openstack-puppet/modules/openstacklib/lib/puppet/provider/openstack.rb:87:in `loop'

2020-04-18T03:10:32.032 /usr/share/openstack-puppet/modules/openstacklib/lib/puppet/provider/openstack.rb:87:in `block in request'

2020-04-18T03:10:32.039 /usr/share/ruby/vendor_ruby/puppet/util.rb:128:in `withenv'

2020-04-18T03:10:32.043 /usr/share/openstack-puppet/modules/openstacklib/lib/puppet/provider/openstack.rb:82:in `request'

2020-04-18T03:10:32.049 /usr/share/openstack-puppet/modules/keystone/lib/puppet/provider/keystone.rb:325:in `request_by_service_token'

2020-04-18T03:10:32.055 /usr/share/openstack-puppet/modules/keystone/lib/puppet/provider/keystone.rb:258:in `rescue in request'

2020-04-18T03:10:32.061 /usr/share/openstack-puppet/modules/keystone/lib/puppet/provider/keystone.rb:254:in `request'

2020-04-18T03:10:32.067 /usr/share/openstack-puppet/modules/keystone/lib/puppet/provider/keystone_role/openstack.rb:51:in `instances'

2020-04-18T03:10:32.073 /usr/share/openstack-puppet/modules/keystone/lib/puppet/provider/keystone_role/openstack.rb:64:in `prefetch'

2020-04-18T03:10:32.079 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:323:in `prefetch'

2020-04-18T03:10:32.085 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:223:in `prefetch_if_necessary'

2020-04-18T03:10:32.091 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:107:in `block in evaluate'

2020-04-18T03:10:32.096 /usr/share/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:116:in `call'

2020-04-18T03:10:32.102 /usr/share/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:116:in `traverse'

2020-04-18T03:10:32.108 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:154:in `evaluate'

2020-04-18T03:10:32.114 /usr/share/ruby/vendor_ruby/puppet/resource/catalog.rb:222:in `block in apply'

2020-04-18T03:10:32.121 /usr/share/ruby/vendor_ruby/puppet/util/log.rb:155:in `with_destination'

2020-04-18T03:10:32.126 /usr/share/ruby/vendor_ruby/puppet/transaction/report.rb:142:in `as_logging_destination'

2020-04-18T03:10:32.132 /usr/share/ruby/vendor_ruby/puppet/resource/catalog.rb:221:in `apply'

2020-04-18T03:10:32.138 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:171:in `block in apply_catalog'

2020-04-18T03:10:32.143 /usr/share/ruby/vendor_ruby/puppet/util.rb:223:in `block in benchmark'

2020-04-18T03:10:32.148 /usr/share/ruby/benchmark.rb:296:in `realtime'

2020-04-18T03:10:32.154 /usr/share/ruby/vendor_ruby/puppet/util.rb:222:in `benchmark'

2020-04-18T03:10:32.160 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:170:in `apply_catalog'

2020-04-18T03:10:32.166 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:343:in `run_internal'

2020-04-18T03:10:32.171 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:221:in `block in run'

2020-04-18T03:10:32.176 /usr/share/ruby/vendor_ruby/puppet/context.rb:65:in `override'

2020-04-18T03:10:32.182 /usr/share/ruby/vendor_ruby/puppet.rb:241:in `override'

2020-04-18T03:10:32.188 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:195:in `run'

2020-04-18T03:10:32.193 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:350:in `apply_catalog'

2020-04-18T03:10:32.199 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:274:in `block in main'

2020-04-18T03:10:32.204 /usr/share/ruby/vendor_ruby/puppet/context.rb:65:in `override'

2020-04-18T03:10:32.210 /usr/share/ruby/vendor_ruby/puppet.rb:241:in `override'

2020-04-18T03:10:32.216 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:225:in `main'

2020-04-18T03:10:32.222 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:170:in `run_command'

2020-04-18T03:10:32.229 /usr/share/ruby/vendor_ruby/puppet/application.rb:344:in `block in run'

2020-04-18T03:10:32.234 /usr/share/ruby/vendor_ruby/puppet/util.rb:540:in `exit_on_fail'

2020-04-18T03:10:32.240 /usr/share/ruby/vendor_ruby/puppet/application.rb:344:in `run'

2020-04-18T03:10:32.246 /usr/share/ruby/vendor_ruby/puppet/util/command_line.rb:132:in `run'

2020-04-18T03:10:32.251 /usr/share/ruby/vendor_ruby/puppet/util/command_line.rb:72:in `execute'

2020-04-18T03:10:32.256 /usr/bin/puppet:5:in `<main>'

==========================================================================================

thanks,

yours,

zhou qingyang.

At 2020-04-19 13:36:53, "Sun, Austin" <austin.sun at intel.com> wrote:

Hi qingyang:

     Would you like to share some more info?

1) are u install in bare metal or virtual environment ?  how much are memory and disk ?

2) could you provide localhost.yml you are using ?

3) provide /var/log/puppet/latest/puppet.log  ?

Thanks.

BR
Austin Sun.

From:周擎阳 <18245043314 at 163.com>
Sent: Saturday, April 18, 2020 6:52 PM
To:starlingx-discuss at lists.starlingx.io
Subject: [Starlingx-discuss] error when installing starlingx3.0 all-in-one in bare metal

Dear sir:

   I am trying to install starlingx following the all-in-one link:https://docs.starlingx.io/deploy_install_guides/r3_release/bare_metal/aio_simplex.html. However I stuck in the "ansible" step. From the dmesg, I can see the errors below:

[79591.931815] block drbd0: Resize while not connected was forced by the user!

[79592.058071] block drbd8: local WRITE IO error sector 1296+4088 on dm-8

[79592.058083] block drbd8: disk( UpToDate -> Failed ) 

[79592.058113] block drbd8: Local IO failed in __req_mod. Detaching...

[79592.058146] block drbd8: IO ERROR: neither local nor remote data, sector 0+0

[79592.058178] block drbd8: helper command: /sbin/drbdadm pri-on-incon-degr minor-8

[79592.061512] block drbd8: IO ERROR: neither local nor remote data, sector 5384+4096

[79592.061603] drbd8: WRITE SAME failed. Manually zeroing.

[79592.061664] block drbd8: IO ERROR: neither local nor remote data, sector 5384+256

[79592.061764] block drbd8: IO ERROR: neither local nor remote data, sector 5640+256

[79592.061859] block drbd8: IO ERROR: neither local nor remote data, sector 5896+256

[79592.061880] block drbd8: helper command: /sbin/drbdadm pri-on-incon-degr minor-8 exit code 0 (0x0)

[79592.061972] block drbd8: 1024 MB (262127 bits) marked out-of-sync by on disk bit-map.

[79592.061988] block drbd8: disk( Failed -> Diskless ) 

[79592.835252] block drbd0: local WRITE IO error sector 2328+4080 on dm-3

[79592.835262] block drbd0: disk( UpToDate -> Failed ) 

[79592.835293] block drbd0: Local IO failed in __req_mod. Detaching...

[79592.835369] drbd0: WRITE SAME failed. Manually zeroing.

[79592.835388] block drbd0: helper command: /sbin/drbdadm pri-on-incon-degr minor-0

[79592.838928] block drbd0: helper command: /sbin/drbdadm pri-on-incon-degr minor-0 exit code 0 (0x0)

[79592.838970] block drbd0: 2048 MB (524263 bits) marked out-of-sync by on disk bit-map.

[79592.838980] block drbd0: disk( Failed -> Diskless ) 

[79593.038394] EXT4-fs error (device drbd0): ext4_wait_block_bitmap:516: comm initdb: Cannot read block bitmap - block_group = 1, block_bitmap = 258

[79593.038541] Buffer I/O error on dev drbd0, logical block 0, lost sync page write

[79593.038686] EXT4-fs warning (device drbd0): ext4_end_bio:302: I/O error -5 writing to inode 29 (offset 0 size 20480 starting block 65536)

[79593.038689] Buffer I/O error on device drbd0, logical block 65536

[79593.038743] Buffer I/O error on device drbd0, logical block 65537

[79593.038805] Buffer I/O error on device drbd0, logical block 65538

[79593.038858] Buffer I/O error on device drbd0, logical block 65539

[79593.038911] Buffer I/O error on device drbd0, logical block 65540

[79595.501408] block drbd2: Resize while not connected was forced by the user!

[79595.701486] EXT4-fs (drbd5): mounted filesystem with ordered data mode. Opts: (null)

[79595.731488] EXT4-fs (drbd5): re-mounted. Opts: (null)

[79595.755858] block drbd5: Resize while not connected was forced by the user!

[79596.382427] EXT4-fs (drbd1): mounted filesystem with ordered data mode. Opts: (null)

[79596.427131] EXT4-fs (drbd1): re-mounted. Opts: (null)

[79596.456439] block drbd1: Resize while not connected was forced by the user!

[79596.763258] block drbd1: local WRITE IO error sector 2320+4088 on dm-9

[79596.763270] block drbd1: disk( UpToDate -> Failed ) 

[79596.763304] block drbd1: Local IO failed in __req_mod. Detaching...

[79596.763412] block drbd1: helper command: /sbin/drbdadm pri-on-incon-degr minor-1

[79596.767304] block drbd1: helper command: /sbin/drbdadm pri-on-incon-degr minor-1 exit code 0 (0x0)

[79596.767348] block drbd1: 2048 MB (524263 bits) marked out-of-sync by on disk bit-map.

[79596.767358] block drbd1: disk( Failed -> Diskless ) 

[79596.788531] drbd1: WRITE SAME failed. Manually zeroing.

[79597.168463] block drbd7: role( Secondary -> Primary ) disk( Inconsistent -> UpToDate ) 

[79597.179035] block drbd7: Forced to consider local data as UpToDate!

[79597.179053] block drbd7: new current UUID 9759105C28A7AAAD:0000000000000004:0000000000000000:0000000000000000

[79597.616990] block drbd0: 52 messages suppressed in /builddir/build/BUILD/drbd-8.4.11-1/obj/default/drbd_req.c:1446.

[79597.616999] block drbd0: IO ERROR: neither local nor remote data, sector 1572864+8

[79597.617096] Buffer I/O error on dev drbd0, logical block 196608, lost sync page write

[79597.617187] JBD2: Error -5 detected when updating journal superblock for drbd0-8.

[79597.617274] Aborting journal on device drbd0-8.

[79597.617328] Buffer I/O error on dev drbd0, logical block 196608, lost sync page write

[79597.617417] JBD2: Error -5 detected when updating journal superblock for drbd0-8.

[79597.680678] Buffer I/O error on dev drbd8, logical block 65536, lost sync page write

[79597.680772] JBD2: Error -5 detected when updating journal superblock for drbd8-8.

[79597.680859] Aborting journal on device drbd8-8.

[79597.680914] Buffer I/O error on dev drbd8, logical block 65536, lost sync page write

[79597.681003] JBD2: Error -5 detected when updating journal superblock for drbd8-8.

[79597.760212] EXT4-fs (drbd0): previous I/O error to superblock detected

[79597.760337] Buffer I/O error on dev drbd0, logical block 0, lost sync page write

[79597.760450] EXT4-fs error (device drbd0): ext4_journal_check_start:56: Detected aborted journal

[79597.760551] EXT4-fs (drbd0): Remounting filesystem read-only

[79597.760620] EXT4-fs (drbd0): previous I/O error to superblock detected

[79597.760682] Buffer I/O error on dev drbd0, logical block 0, lost sync page write

[79600.177668] block drbd5: local WRITE IO error sector 1296+4088 on dm-2

[79600.177680] block drbd5: disk( UpToDate -> Failed ) 

[79600.177711] block drbd5: Local IO failed in __req_mod. Detaching...

[79600.177786] drbd5: WRITE SAME failed. Manually zeroing.

[79600.177813] block drbd5: helper command: /sbin/drbdadm pri-on-incon-degr minor-5

[79600.181353] block drbd5: helper command: /sbin/drbdadm pri-on-incon-degr minor-5 exit code 0 (0x0)

[79600.181396] block drbd5: 1024 MB (262127 bits) marked out-of-sync by on disk bit-map.

[79600.181406] block drbd5: disk( Failed -> Diskless ) 

[79601.244660] EXT4-fs error (device drbd1): ext4_wait_block_bitmap:516: comm async_310: Cannot read block bitmap - block_group = 1, block_bitmap = 258

[79601.244825] Buffer I/O error on dev drbd1, logical block 0, lost sync page write

[79601.244989] EXT4-fs warning (device drbd1): ext4_end_bio:302: I/O error -5 writing to inode 17 (offset 0 size 4096 starting block 65536)

[79601.244994] Buffer I/O error on device drbd1, logical block 65536

[79601.607395] Buffer I/O error on dev drbd1, logical block 196608, lost sync page write

[79601.607495] JBD2: Error -5 detected when updating journal superblock for drbd1-8.

[79601.607582] Aborting journal on device drbd1-8.

[79601.607638] Buffer I/O error on dev drbd1, logical block 196608, lost sync page write

[79601.607727] JBD2: Error -5 detected when updating journal superblock for drbd1-8.

[79602.351833] EXT4-fs (drbd1): previous I/O error to superblock detected

[79602.351905] block drbd1: 28 messages suppressed in /builddir/build/BUILD/drbd-8.4.11-1/obj/default/drbd_req.c:1446.

[79602.351909] block drbd1: IO ERROR: neither local nor remote data, sector 0+8

[79602.351970] Buffer I/O error on dev drbd1, logical block 0, lost sync page write

[79602.352063] EXT4-fs error (device drbd1): ext4_journal_check_start:56: Detected aborted journal

[79602.352159] EXT4-fs (drbd1): Remounting filesystem read-only

[79602.352216] EXT4-fs (drbd1): previous I/O error to superblock detected

[79602.352274] Buffer I/O error on dev drbd1, logical block 0, lost sync page write

[79605.662600] Buffer I/O error on dev drbd5, logical block 65536, lost sync page write

[79605.662694] JBD2: Error -5 detected when updating journal superblock for drbd5-8.

[79605.662781] Aborting journal on device drbd5-8.

[79605.662836] Buffer I/O error on dev drbd5, logical block 65536, lost sync page write

[79605.662924] JBD2: Error -5 detected when updating journal superblock for drbd5-8.

[80656.347102] block drbd8: 3 messages suppressed in /builddir/build/BUILD/drbd-8.4.11-1/obj/default/drbd_req.c:1446.

[80656.347111] block drbd8: IO ERROR: neither local nor remote data, sector 1096+8

[80656.347330] EXT4-fs warning (device drbd8): __ext4_read_dirblock:902: error reading directory block (ino 2, block 0)

[80656.347372] block drbd8: IO ERROR: neither local nor remote data, sector 0+8

[80656.347586] Buffer I/O error on dev drbd8, logical block 0, lost sync page write

[80656.347798] EXT4-fs error (device drbd8): ext4_journal_check_start:56: Detected aborted journal

[80656.348118] EXT4-fs (drbd8): Remounting filesystem read-only

[80656.348130] EXT4-fs (drbd8): previous I/O error to superblock detected

[80656.348135] block drbd8: IO ERROR: neither local nor remote data, sector 0+8

[80656.348140] Buffer I/O error on dev drbd8, logical block 0, lost sync page write

[80661.007923] block drbd8: IO ERROR: neither local nor remote data, sector 1096+8

[80661.008142] EXT4-fs warning (device drbd8): __ext4_read_dirblock:902: error reading directory block (ino 2, block 0)

[80663.424845] block drbd8: IO ERROR: neither local nor remote data, sector 1096+8

[80663.425063] EXT4-fs warning (device drbd8): __ext4_read_dirblock:902: error reading directory block (ino 2, block 0)

[80670.001525] block drbd8: IO ERROR: neither local nor remote data, sector 1096+8

[80670.001751] EXT4-fs warning (device drbd8): __ext4_read_dirblock:902: error reading directory block (ino 2, block 0)

[80817.182038] block drbd8: IO ERROR: neither local nor remote data, sector 1096+8

[80817.182271] EXT4-fs error (device drbd8): ext4_find_entry:1312: inode #2: comm sh: reading directory lblock 0

[80817.182529] EXT4-fs (drbd8): previous I/O error to superblock detected

[80817.182723] block drbd8: IO ERROR: neither local nor remote data, sector 0+8

[80817.182956] Buffer I/O error on dev drbd8, logical block 0, lost sync page write

[80817.186801] block drbd8: IO ERROR: neither local nor remote data, sector 1096+8

[80817.187036] EXT4-fs error (device drbd8): ext4_find_entry:1312: inode #2: comm sh: reading directory lblock 0

[80817.187271] EXT4-fs (drbd8): previous I/O error to superblock detected

[80817.187457] block drbd8: IO ERROR: neither local nor remote data, sector 0+8

[80817.187648] Buffer I/O error on dev drbd8, logical block 0, lost sync page write

I guess there are something wrong with drbd, but I am not sure about the problem. Do you have any ideas or do you need more information to solve it?

Thanks.

yours,

zhou qingyang.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20200419/f30e9497/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 122302 bytes
Desc: not available
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20200419/f30e9497/attachment-0001.png>