clickhouse 出现 strerror: Structure needs cleaning 错误

发布于 2 个月前 作者 qiuzhilin 245 次浏览 来自 问答

系统:centos7.x 表引擎: ReplacingMergeTree

异常信息 2021.07.14 15:57:16.499995 [ 64830 ] {} <Error> auto DB::IBackgroundJobExecutor::jobExecutingTask()::(anonymous class)::operator()() const: Code: 76, e.displayText() = DB::ErrnoException: Cannot open file /var/lib/clickhouse/store/056/0560e033-1663-49e1-80a8-5f448ae7fb0a/tmp_merge_202107_1_5806_2020/ProfileEvent_ZooKeeperInit.bin, errno: 117, strerror: Structure needs cleaning, Stack trace (when copying this message, always include the lines below):

  1. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87f714a in /usr/bin/clickhouse
  2. DB::throwFromErrnoWithPath(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) @ 0x87f80e9 in /usr/bin/clickhouse
  3. DB::WriteBufferFromFile::WriteBufferFromFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, unsigned long, int, unsigned int, char*, unsigned long) @ 0x88ee83e in /usr/bin/clickhouse
  4. DB::DiskLocal::writeFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, unsigned long, DB::WriteMode) @ 0xefee2a5 in /usr/bin/clickhouse
  5. DB::MergeTreeDataPartWriterOnDisk::Stream::Stream(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptrDB::IDisk, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptrDB::ICompressionCodec const&, unsigned long) @ 0xfc96480 in /usr/bin/clickhouse
  6. ? @ 0xfc9f69b in /usr/bin/clickhouse
  7. ? @ 0xef41784 in /usr/bin/clickhouse
  8. DB::MergeTreeDataPartWriterWide::addStreams(DB::NameAndTypePair const&, std::__1::shared_ptrDB::IAST const&) @ 0xfc9a207 in /usr/bin/clickhouse
  9. DB::MergeTreeDataPartWriterWide::MergeTreeDataPartWriterWide(std::__1::shared_ptr<DB::IMergeTreeDataPart const> const&, DB::NamesAndTypesList const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::vector<std::__1::shared_ptr<DB::IMergeTreeIndex const>, std::__1::allocator<std::__1::shared_ptr<DB::IMergeTreeIndex const> > > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptrDB::ICompressionCodec const&, DB::MergeTreeWriterSettings const&, DB::MergeTreeIndexGranularity const&) @ 0xfc9a013 in /usr/bin/clickhouse
  10. DB::MergeTreeDataPartWide::getWriter(DB::NamesAndTypesList const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::vector<std::__1::shared_ptr<DB::IMergeTreeIndex const>, std::__1::allocator<std::__1::shared_ptr<DB::IMergeTreeIndex const> > > const&, std::__1::shared_ptrDB::ICompressionCodec const&, DB::MergeTreeWriterSettings const&, DB::MergeTreeIndexGranularity const&) const @ 0xfc8a670 in /usr/bin/clickhouse
  11. DB::MergedBlockOutputStream::MergedBlockOutputStream(std::__1::shared_ptr<DB::IMergeTreeDataPart const> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::NamesAndTypesList const&, std::__1::vector<std::__1::shared_ptr<DB::IMergeTreeIndex const>, std::__1::allocator<std::__1::shared_ptr<DB::IMergeTreeIndex const> > > const&, std::__1::shared_ptrDB::ICompressionCodec, bool) @ 0xfd66d0a in /usr/bin/clickhouse
  12. DB::MergeTreeDataMergerMutator::mergePartsToTemporaryPart(DB::FutureMergedMutatedPart const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::BackgroundProcessListEntry<DB::MergeListElement, DB::MergeInfo>&, std::__1::shared_ptrDB::RWLockImpl::LockHolderImpl&, long, DB::Context const&, std::__1::unique_ptr<DB::IReservation, std::__1::default_deleteDB::IReservation > const&, bool, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0xfc4c440 in /usr/bin/clickhouse
  13. DB::StorageMergeTree::mergeSelectedParts(std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, bool, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, DB::StorageMergeTree::MergeMutateSelectedEntry&, std::__1::shared_ptrDB::RWLockImpl::LockHolderImpl&) @ 0xf9ab718 in /usr/bin/clickhouse
  14. ? @ 0xf9bda7c in /usr/bin/clickhouse
  15. ? @ 0xfb90b47 in /usr/bin/clickhouse
  16. ThreadPoolImpl<ThreadFromGlobalPool>::worker(std::__1::__list_iterator<ThreadFromGlobalPool, void*>) @ 0x8839d78 in /usr/bin/clickhouse
  17. ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::‘lambda1’()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::‘lambda1’()&&…)::‘lambda’()::operator()() @ 0x883bd3f in /usr/bin/clickhouse
  18. ThreadPoolImplstd::__1::thread::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x88372bf in /usr/bin/clickhouse
  19. ? @ 0x883ade3 in /usr/bin/clickhouse
  20. start_thread @ 0x7dd5 in /usr/lib64/libpthread-2.17.so
  21. __clone @ 0xfdead in /usr/lib64/libc-2.17.so (version 21.4.6.55 (official build))
1 回复

此异常的出现伴随着,system.metric_log flush 到磁盘 的动作,应该跟system.metric_log 的flush 有关

回到顶部