Discussion:
[PATCH] makedumpfile: Error on re-filtering the dump file with no free pages
Zaslonko Mikhail
2017-05-17 15:02:24 UTC
Permalink
Please find the enclosed patch below.
Atsushi Kumagai
2017-05-18 04:37:20 UTC
Permalink
Hello Zaslonko,
When re-filtering the dump file after the free pages have already been
stripped out we get an error "Can't get necessary symbols for excluding
free pages" if newly specified dump level is below 16 (preserves free
pages).
According to the code, the check for the new dump level is done BEFORE
the new dump level is actually set (based on the dump level specified in
the command and the one from the input dump file).
Moving the new_dump_level calculation ahead would fix the error.
Well, I guess the real problem is different.
The error you said is printed by exclude_free_page():

if ((SYMBOL(node_data) == NOT_FOUND_SYMBOL)
&& (SYMBOL(pgdat_list) == NOT_FOUND_SYMBOL)
&& (SYMBOL(contig_page_data) == NOT_FOUND_SYMBOL)) {
ERRMSG("Can't get necessary symbols for excluding free pages.\n");
return FALSE;
}

I think the availability of these symbols are not related to free pages.
exclude_free_page() is called if info->page_is_buddy is null, I estimate that
this situation occurs only if the kernel is older(2.6.16 or before).

However, I don't think you use such too old kernel, so I suspect that
setup_page_is_buddy() should be updated for newer kernel.
Could you tell me your kernel version and how to reproduce this issue ?


Thanks,
Atsushi Kumagai
---
makedumpfile.c | 34 ++++++++++++++++++++++------------
1 file changed, 22 insertions(+), 12 deletions(-)
diff --git a/makedumpfile.c b/makedumpfile.c
index e69b6df..24f99fc 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -9774,10 +9774,25 @@ writeout_multiple_dumpfiles(void)
return ret;
}
+void
+update_dump_level(void)
+{
+ int new_level;
+
+ new_level = info->dump_level | info->kh_memory->dump_level;
+ if (new_level != info->dump_level) {
+ info->dump_level = new_level;
+ MSG("dump_level is changed to %d, " \
+ "because %s was created by dump_level(%d).",
+ new_level, info->name_memory,
+ info->kh_memory->dump_level);
+ }
+}
+
int
create_dumpfile(void)
{
- int num_retry, status, new_level;
+ int num_retry, status;
if (!open_files_for_creating_dumpfile())
return FALSE;
@@ -9786,6 +9801,10 @@ create_dumpfile(void)
if (!get_elf_info(info->fd_memory, info->name_memory))
return FALSE;
}
+
+ if (info->flag_refiltering)
+ update_dump_level();
+
if (!initial())
return FALSE;
@@ -9804,17 +9823,8 @@ create_dumpfile(void)
num_retry = 0;
- if (info->flag_refiltering) {
- /* Change dump level */
- new_level = info->dump_level | info->kh_memory->dump_level;
- if (new_level != info->dump_level) {
- info->dump_level = new_level;
- MSG("dump_level is changed to %d, " \
- "because %s was created by dump_level(%d).",
- new_level, info->name_memory,
- info->kh_memory->dump_level);
- }
- }
+ if (info->flag_refiltering)
+ update_dump_level();
if ((info->name_filterconfig || info->name_eppic_config)
&& !gather_filter_info())
--
1.8.3.1
Michael Holzheu
2017-05-18 12:02:54 UTC
Permalink
Am Thu, 18 May 2017 04:37:20 +0000
Post by Atsushi Kumagai
Hello Zaslonko,
When re-filtering the dump file after the free pages have already been
stripped out we get an error "Can't get necessary symbols for excluding
free pages" if newly specified dump level is below 16 (preserves free
pages).
According to the code, the check for the new dump level is done BEFORE
the new dump level is actually set (based on the dump level specified in
the command and the one from the input dump file).
Moving the new_dump_level calculation ahead would fix the error.
Well, I guess the real problem is different.
if ((SYMBOL(node_data) == NOT_FOUND_SYMBOL)
&& (SYMBOL(pgdat_list) == NOT_FOUND_SYMBOL)
&& (SYMBOL(contig_page_data) == NOT_FOUND_SYMBOL)) {
ERRMSG("Can't get necessary symbols for excluding free pages.\n");
return FALSE;
}
I think the availability of these symbols are not related to free pages.
exclude_free_page() is called if info->page_is_buddy is null, I estimate that
this situation occurs only if the kernel is older(2.6.16 or before).
However, I don't think you use such too old kernel, so I suspect that
setup_page_is_buddy() should be updated for newer kernel.
Mikhail is on vacation now - so I try to explain:

The test case is as follows:

1) We have a -d31 filtered dump "dump.d31"
2) We want to compress the dump with "makedumpfile -c dump.31 dump31.compressed"

This fails with:

makedumpfile -c dump.31 dump.31.compressed
Excluding unnecessary pages : [100.0 %]
exclude_free_page: Can't get necessary symbols for excluding free pages.
dump_level is changed to 31, because dump.31 was created by dump_level(31).
makedumpfile Failed.

The problem is that setup_page_is_buddy() is not called in this case because
info->dump_level is still 0 since it was not adjusted early enough:

if (info->dump_level & DL_EXCLUDE_FREE)
setup_page_is_buddy();

Because it is not set info->page_is_buddy is NULL and therefore the following
if condition gets true:

if ((info->dump_level & DL_EXCLUDE_FREE) && !info->page_is_buddy)
if (!exclude_free_page(cycle))
return FALSE;

Since we don't have the symbols in VMCOREINFO (and IMHO don't need it?) the
exclude_free_page() functions fails with the described error message.

So our fix is to adjust the info->level before setup_page_is_buddy() is called.

Best Regards
Michael
Post by Atsushi Kumagai
Could you tell me your kernel version and how to reproduce this issue ?
Atsushi Kumagai
2017-05-19 02:35:40 UTC
Permalink
Post by Michael Holzheu
Post by Atsushi Kumagai
Hello Zaslonko,
When re-filtering the dump file after the free pages have already been
stripped out we get an error "Can't get necessary symbols for excluding
free pages" if newly specified dump level is below 16 (preserves free
pages).
According to the code, the check for the new dump level is done BEFORE
the new dump level is actually set (based on the dump level specified in
the command and the one from the input dump file).
Moving the new_dump_level calculation ahead would fix the error.
Well, I guess the real problem is different.
if ((SYMBOL(node_data) == NOT_FOUND_SYMBOL)
&& (SYMBOL(pgdat_list) == NOT_FOUND_SYMBOL)
&& (SYMBOL(contig_page_data) == NOT_FOUND_SYMBOL)) {
ERRMSG("Can't get necessary symbols for excluding free pages.\n");
return FALSE;
}
I think the availability of these symbols are not related to free pages.
exclude_free_page() is called if info->page_is_buddy is null, I estimate that
this situation occurs only if the kernel is older(2.6.16 or before).
However, I don't think you use such too old kernel, so I suspect that
setup_page_is_buddy() should be updated for newer kernel.
Thanks for your explanation, I understand this issue properly.
Post by Michael Holzheu
1) We have a -d31 filtered dump "dump.d31"
2) We want to compress the dump with "makedumpfile -c dump.31 dump31.compressed"
makedumpfile -c dump.31 dump.31.compressed
Excluding unnecessary pages : [100.0 %]
exclude_free_page: Can't get necessary symbols for excluding free pages.
dump_level is changed to 31, because dump.31 was created by dump_level(31).
makedumpfile Failed.
The problem is that setup_page_is_buddy() is not called in this case because
if (info->dump_level & DL_EXCLUDE_FREE)
setup_page_is_buddy();
Because it is not set info->page_is_buddy is NULL and therefore the following
if ((info->dump_level & DL_EXCLUDE_FREE) && !info->page_is_buddy)
if (!exclude_free_page(cycle))
return FALSE;
Since we don't have the symbols in VMCOREINFO (and IMHO don't need it?) the
exclude_free_page() functions fails with the described error message.
It seems that it's better if I update the condition check of exclude_free_page()
for recent kernels, but those symbols are unnecessary in this case as you thought
anyway. exclude_free_page() shouldn't be called for recent kernels, I don't think
this is an actual problem.
Post by Michael Holzheu
So our fix is to adjust the info->level before setup_page_is_buddy() is called.
I'm sure this fix is reasonable, I'll merge this into v1.6.2.

Thanks,
Atsushi Kumagai
Michael Holzheu
2017-05-19 07:27:45 UTC
Permalink
Am Fri, 19 May 2017 02:35:40 +0000
Post by Atsushi Kumagai
Post by Michael Holzheu
Post by Atsushi Kumagai
Hello Zaslonko,
[snip]
Post by Atsushi Kumagai
Post by Michael Holzheu
Since we don't have the symbols in VMCOREINFO (and IMHO don't need it?) the
exclude_free_page() functions fails with the described error message.
It seems that it's better if I update the condition check of exclude_free_page()
for recent kernels, but those symbols are unnecessary in this case as you thought
anyway. exclude_free_page() shouldn't be called for recent kernels, I don't think
this is an actual problem.
Post by Michael Holzheu
So our fix is to adjust the info->level before setup_page_is_buddy() is called.
I'm sure this fix is reasonable, I'll merge this into v1.6.2.
Thanks!

Michael

Loading...